Display system, display device, and display control method

Abstract

The display system (100) comprises: an acquisition device (20) for acquiring speed information indicating the speed of a moving object; a detection device (40) for detecting the amount of posture change of the moving object; a display processing device (30) for controlling the display position of an image based on a reference position and a correction amount; and a correction processing device (50) for setting the correction amount based on the speed and the amount of posture change. The correction processing device (50) infers whether the correction amount is greater than a predetermined amount based on the speed and the amount of posture change, and adjusts the correction amount based on the inference result.

Description

Technical Field

[0001] The present disclosure relates to a display system, a display device, and a display control method for controlling a display position of an image according to the motion of a moving object. Background Art

[0002] Patent document 1 discloses a vehicle information display system that uses a head-up display (HUD) device to perform augmented reality (AR) display. The HUD device projects light representing a virtual image on the windshield of the vehicle, allowing an observer who is a passenger in the vehicle to visually confirm the real scene and virtual image of the outside world of the vehicle at the same time. For example, a virtual image representing the guidance path of the vehicle is associated with a display object in the real scene, such as a road, and displayed. As a result, passengers can visually confirm the guidance path while confirming the real scene. The vehicle information display system of Patent Document 1 corrects the display position of the virtual image according to acceleration. As a result, when the vehicle suddenly decelerates or accelerates, the position deviation of the virtual image is suppressed.

[0003] Prior Art Literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2015-101311 Summary of the invention

[0006] -Problems to be solved by the invention-

[0007] If the positional deviation of the image is to be corrected, the correction process may be noticeable to the observer.

[0008] The present disclosure provides a display system, a display device, and a display control method that suppress the positional deviation of an image with high accuracy and improve the quality of the appearance of positional deviation correction.

[0009] -Methods for solving the problem-

[0010] The display system disclosed in the present invention comprises: an acquisition device for acquiring speed information indicating the speed of a moving body; a detection device for detecting the amount of posture change of the moving body; a display processing device for controlling the display position of an image based on a reference position and a correction amount; and a correction processing device for setting a correction amount based on the speed and the amount of posture change, the correction processing device infers whether the correction amount is above a prescribed amount based on the speed and the amount of posture change, and adjusts the correction amount based on the inference result.

[0011] The display device disclosed in the present invention comprises: an acquisition unit, which acquires speed information indicating the speed of a moving body, and change information indicating the amount of change in posture of the moving body; a display unit, which displays an image at a display position based on a reference position and a correction amount; and a control unit, which sets a correction amount based on the speed and the amount of change in posture, the control unit sets a threshold value based on the speed, sets the correction amount based on the amount of change in posture when the amount of change in posture is below the threshold value, and returns the display position to the reference position when the amount of change in posture is greater than the threshold value.

[0012] The display control method disclosed in the present invention is a display control method performed by a computing unit of a computer, comprising: a step of acquiring speed information indicating the speed of a moving body, and change information indicating the change amount of the posture of the moving body; and a step of controlling the display position of an image based on a reference position and a correction amount, wherein the step of controlling the display position comprises: setting a threshold based on the speed, setting a correction amount based on the posture change amount while the posture change amount is below the threshold, and returning the display position to the reference position while the posture change amount is greater than the threshold.

[0013] These general and specific aspects may also be implemented by systems, methods, and computer programs, and combinations thereof.

[0014] -Effects of the Invention-

[0015] The display system disclosed in the present invention can suppress the position deviation of the image with high precision and improve the appearance quality of the position deviation correction. For example, when it is inferred that the correction amount corresponding to the posture change amount is greater than a specified amount, the correction amount is reset. When it is inferred that the correction amount remains less than the specified amount, an adjustment based on the correction is performed, thereby achieving a high-precision position deviation correction that takes into account the correction, reset, and the quality of the appearance at the time of reset. In addition, the display device and display control method disclosed in the present invention set the threshold value of the posture change amount according to the speed of the moving body, so that in the control of the display position based on the posture change amount, the position deviation of the image can be suppressed with high precision. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a diagram for explaining the head-up display (HUD) in the first embodiment.

[0017] Figure 2 This is a block diagram showing the configuration of a display system in the first embodiment.

[0018] Figure 3 This is a diagram showing an example of threshold data.

[0019] Figure 4A It is a graph showing the threshold value according to the vehicle speed.

[0020] Figure 4BThis is another graph showing threshold values ​​according to vehicle speed.

[0021] Figure 5 This is a diagram showing an example of a real scene that can be seen through a windshield.

[0022] Figure 6 This is a diagram showing an example of a virtual image.

[0023] Fig. 7A 2 is a diagram showing a vehicle that is not tilted.

[0024] Figure 7B This is a diagram for explaining an example in which a virtual image is displayed at a reference position when the vehicle is not tilted.

[0025] Fig. 8A 2 is a diagram showing a vehicle in a forward leaning posture.

[0026] Figure 8B This is a diagram for explaining an example of positional deviation of a virtual image that occurs when the vehicle is in a forward leaning posture.

[0027] Fig. 9 This is a diagram for explaining correction of the display position of a virtual image.

[0028] Fig.10 This is a diagram for explaining the positional deviation of a virtual image due to noise of a gyro sensor.

[0029] Fig.11 This is a flowchart showing the display processing in the first embodiment.

[0030] Fig.12 This is a flowchart showing the correction process in the first embodiment.

[0031] Fig.13A This is a diagram for explaining resetting of the correction amount in the first embodiment.

[0032] Fig. 13B This is a diagram for explaining another example of resetting the correction amount in the first embodiment.

[0033] Fig.14 This is a flowchart showing the display processing in the second embodiment.

[0034] Fig.15 This is a flowchart showing the correction process in the second embodiment.

[0035] Fig.16 This is a diagram for explaining the setting of the offset value in the second embodiment.

[0036] Fig.17 This is a flowchart showing the correction process in the third embodiment.

[0037] Fig.18 This is a flowchart showing the correction process in the fourth embodiment.

[0038] Fig.19 This is a block diagram showing the structure of a display system in the fifth embodiment.

[0039] Fig. 20 This is a flowchart showing the correction processing in the fifth embodiment.

[0040] Fig.21A This is a diagram for explaining resetting of the deviation amount in the fifth embodiment.

[0041] Fig. 21B This is a diagram for explaining another example of resetting the offset amount in the fifth embodiment.

[0042] Fig. 22 This is a flowchart showing the correction processing in the sixth embodiment.

[0043] Fig.23 This is a flowchart showing the correction processing in the seventh embodiment.

[0044] Fig.24 It is a block diagram showing the structure of a display device in the eighth embodiment.

[0045] Fig.25 It is an explanatory diagram showing the temporal change of the angle detected by the gyro sensor. DETAILED DESCRIPTION

[0046] (Understanding that is the basis of the present disclosure)

[0047] When the display position of an image is corrected based on the state of the moving object detected based on the output of the sensor, for example, based on the posture of the moving object, a correction error is generated due to the noise of the sensor.

[0048] For example, in order to accurately detect the vibration of a moving body caused by the shape of the bumps and other shapes on the road surface, a gyro sensor is considered. The angles of the moving body around three axes, namely the roll angle, pitch angle, and yaw angle, can be obtained by integrating the angular velocity detected by the gyro sensor. However, due to the characteristics of the device, the angular velocity output by the gyro sensor is not zero even in a stationary state. The so-called drift is generated. Therefore, in the integral calculation of the angular velocity, the error based on the drift is accumulated, and the obtained angle has an error. In this case, if the display position of the image is continuously corrected based on the output of the gyro sensor, the correction error becomes larger. Therefore, if the correction based on the output of the sensor is performed frequently or for a long time, the correction error is accumulated and the display position of the image deviates. For example, in the case of a HUD system that performs an augmented reality (AR) display, the display position of the virtual image may deviate significantly from the specified display object in the real scene, such as a road. Therefore, the observer perceives a sense of disharmony in the display of the image.

[0049] In order to eliminate the accumulated error thus generated, it is conceivable to simply reset the correction amount. The correction amount can be reset by returning the display position of the virtual image to the reference position.

[0050] However, if the correction amount is reset at an arbitrary timing, for example, during position deviation correction, the overlap deviation becomes larger. In addition, if the reset action is visually confirmed, the observer will feel a sense of incongruity. In this way, if the correction amount is simply reset, the correction performance will be reduced and the quality of the appearance will deteriorate.

[0051] The display system, display device, and display control method disclosed herein display an image at a display position based on a reference position and a correction amount set according to the posture change amount while the posture change amount of the moving body is below a threshold value. The display device and display system disclosed herein return the display position of the image to the reference position when the posture change amount is greater than the threshold value. Specifically, the posture change amount is the angular velocity output by the gyro sensor.

[0052] Reference Fig.25 This is to explain that if resetting is performed during position deviation correction, the overlap deviation increases. Fig.25 It is an explanatory diagram showing the temporal change of the angle detected by the gyro sensor. Fig.25 The slope of the graph represents the angular velocity. The angle deviation is detected based on the vibration amplitude of the moving body, and the detected angle deviation is the correction amount. Fig.25As shown in FIG. 1 , there is a limit to the deviation amount that can be corrected for position deviation. When a position deviation exceeds the correction range, the position deviation amount exceeding the correction range cannot be corrected, so the overlap accuracy deteriorates. If the correction amount is reset at the timing when the correction cannot be performed, the deterioration of the overlap accuracy due to the reset can be suppressed. Therefore, the deviation amount exceeding the correction range is detected by comparing with a threshold value and judging. Based on the judgment result, the correction amount is reset, so that the period when the position deviation correction cannot be performed can be effectively utilized.

[0053] In addition, the above-mentioned deviation amount can be determined by using the deviation amount based on the calculated angle, but Fig.25 As shown, it is possible that the judgment is made at the timings Pb1 and Pb2 where the change of the external scenery near the peak values ​​Pk1 and Pk2 of the amplitude is small, and the reset action may be obvious. Therefore, the reset of the correction amount needs to be reset and judged earlier at the stage before the deviation exceeds the correction range and reaches the peak value of the vibration amplitude. Therefore, it is appropriate to use the angular velocity for judgment instead of the deviation based on the angle. For example, based on the angular velocity at the timings Pb1 and Pb2, it is compared with the threshold value, so that it can be predicted that the peak values ​​Pk1 and Pk2 after a specified time are included in the correctable range. In this case, the reset action is not performed. In addition, based on the angular velocity at the timings Pb3 and Pb4, it is compared with the threshold value, so that it can be predicted that the peak values ​​Pk3 and Pk4 after a specified time are not included in the correctable range. In this case, the reset action is performed at a timing outside the correctable range. In this way, the reset process can be prevented from being obvious. In addition, at the moment of the timings Pa1 to Pa4, when the angular velocity exceeds the threshold value, it is determined that the change of the external scenery is large, and the reset action can be performed.

[0054] Furthermore, if the threshold value of the determination based on the angular velocity is changed, a more accurate determination (inference) can be made. When the moving body moves at a high speed, the vibration frequency of the moving body becomes higher, so the angular velocity detected by the gyro sensor is larger even with a smaller amplitude than when the moving body is traveling at a low speed. Therefore, when the moving body is moving at a high speed, the angular velocity is likely to become larger than the threshold value, for example, even when the deviation amount is within the correction range and the position deviation correction can be performed, it is reset due to an erroneous determination, and the display position of the image is returned to the reference position more often. Therefore, for example, when the moving body is moving at a high speed, the display position is hardly corrected, which may cause a decrease in correction performance and deterioration in the quality of the appearance. On the other hand, when the moving body is moving at a low speed, the vibration frequency of the moving body becomes lower, so the angular velocity detected by the gyro sensor is smaller even with a larger amplitude than when the moving body is traveling at a high speed. Therefore, when the moving body is moving at a low speed, the angular velocity is likely to become smaller than the threshold value, for example, the display position correction is often performed. Therefore, for example, when the moving body is moving at a low speed, the display position of the image is hardly returned to the reference position. Furthermore, since it is not possible to detect that the deviation exceeds the correction range, the opportunity for resetting is lost, and errors due to integration are accumulated, which may reduce the correctability.

[0055] Therefore, the display system, display device, and display control method of the present disclosure set a threshold value of the angular velocity for determining whether to return the display position to the reference position according to the speed of the moving body. Specifically, the faster the speed, the greater the threshold value. Thus, it is possible to determine (infer) the vibration amplitude (deviation) according to the speed and angular velocity of the moving body, implement position deviation correction when the vibration amplitude is within the correction range, and reset the accumulated error of the correction value when the change of the external scene is large when the vibration amplitude exceeds the correction range and the position deviation correction cannot be performed. Thus, while maintaining the accuracy of the position deviation correction, it is possible to perform a position deviation correction of the image with excellent accuracy that takes into account the elimination of the accumulated error and the quality of the appearance. For example, even when the moving body moves at a high speed, the display position can be corrected. In addition, even when the moving body moves at a low speed, the display position can be returned to the reference position. By changing the display position according to the correction amount based on the angular velocity, the position deviation of the image based on the posture change can be suppressed with excellent accuracy. In addition, by returning the display position to the reference position, the correction error caused by the drift of the gyro sensor can be reduced. In the present disclosure, “returning the display position to the reference position” includes both completely returning the display position to the reference position and partially returning the display position by bringing the display position closer to the reference position.

[0056] (First embodiment)

[0057] The first embodiment is described below with reference to the accompanying drawings. In the first embodiment, a case where the moving body is a vehicle such as a car and the display system is a head-up display (HUD) system that displays a virtual image in front of the windshield of the vehicle is described as an example. In the first embodiment, while the angular velocity of the vehicle is greater than the first threshold value, the correction amount of the display position is reset to zero. Thus, the display position of the virtual image is returned to the reference position. The angular velocity of the vehicle is an example of the amount of change in the posture of the vehicle.

[0058] 1. Display system structure

[0059] Reference Figures 1 to 3 , the structure of the display system of this embodiment is described.

[0060] Figure 1 is a diagram for explaining HUD. Figure 1 In the diagram, the roll axis of the vehicle 200 is set as the X axis, the pitch axis of the vehicle 200 is set as the Y axis, and the yaw axis of the vehicle 200 is set as the Z axis. That is, the X axis is an axis orthogonal to the Y axis and the Z axis and along the line of sight of the passenger D who visually recognizes the virtual image Iv. The Y axis is an axis along the left-right direction from the perspective of the passenger D who visually recognizes the virtual image Iv. The Z axis is an axis along the height direction of the vehicle 200.

[0061] The display system 100 of the present embodiment is a HUD system that performs so-called augmented reality (AR) display by superimposing a virtual image Iv on the real scene in front of the windshield 210 of the vehicle 200. The virtual image Iv represents specified information. For example, the virtual image Iv is a graphic and text representing a path for guiding to the destination, an expected time of arrival at the destination, a traveling direction, a speed, various warnings, etc. The display system 100 is arranged in the vehicle 200, and projects a display light Lc representing the virtual image Iv into a display area 220 of the windshield 210 of the vehicle 200. In the present embodiment, the display area 220 is an area of ​​a part of the windshield 210. In addition, the display area 220 can also be the entire area of ​​the windshield 210. The display light Lc passes through the windshield 210 and is reflected in the direction of the vehicle. As a result, the passenger D in the vehicle 200 visually recognizes the reflected display light Lc as a virtual image Iv in front of the vehicle 200.

[0062] The display system 100 includes a projection device 10 , an information acquisition device 20 , a display processing device 30 , a posture detection device 40 , and a correction processing device 50 .

[0063] The projection device 10 projects the display light Lc representing the virtual image Iv into the display area 220. The projection device 10 includes, for example, a liquid crystal display element that displays the image of the virtual image Iv, a light source such as an LED that illuminates the liquid crystal display element, and a reflector and a lens that reflect the display light Lc of the image displayed by the liquid crystal display element into the display area 220. The projection device 10 is, for example, provided in the instrument panel of the vehicle 200.

[0064] The information acquisition device 20 acquires information indicating the position of the vehicle, the conditions outside the vehicle, and the speed of the vehicle 200 traveling on the road. Specifically, the information acquisition device 20 measures the position of the vehicle 200 and generates position information indicating the position. The information acquisition device 20 generates vehicle-outside information indicating an object and the distance to the object. The object is a person, a sign, a road, etc. The information acquisition device 20 detects the speed of the vehicle 200 and generates speed information indicating the speed of the vehicle 200. The information acquisition device 20 outputs the position information, vehicle-outside information, and speed information of the vehicle 200.

[0065] The display processing device 30 controls the display of the virtual image Iv based on the position information of the vehicle 200 and the vehicle exterior information obtained from the information acquisition device 20, and outputs the image data of the virtual image Iv to the projection device 10. The display processing device 30 outputs the speed information of the vehicle 200 obtained from the information acquisition device 20 to the correction processing device 50, and obtains the correction amount of the display position of the virtual image Iv from the correction processing device 50.

[0066] The posture detection device 40 detects the angular velocity of the vehicle 200 and outputs angular velocity information indicating the detected angular velocity.

[0067] The correction processing device 50 calculates the correction amount of the display position of the virtual image Iv based on the angular velocity information of the vehicle 200 output from the posture detection device 40 and the velocity information of the vehicle 200 output from the information acquisition device 20. The correction processing device 50 outputs the calculated correction amount to the display processing device 30.

[0068] Figure 2 is a block diagram showing the internal structure of the display system 100 .

[0069] In this embodiment, the information acquisition device 20 includes a GPS (Global Positioning System) module 21 , a camera 22 , and a vehicle speed sensor 23 .

[0070] The GPS module 21 detects the current location of the vehicle 200 in the geographic coordinate system. Specifically, the GPS module 21 receives radio waves from GPS satellites and measures the latitude and longitude of the received location. The GPS module 21 generates location information indicating the measured latitude and longitude.

[0071] The camera 22 captures an external scene and generates captured data. The information acquisition device 20 specifies an object based on the captured data and measures the distance to the object by image processing. The information acquisition device 20 generates information indicating the object and the distance to the object as vehicle exterior information.

[0072] The vehicle speed sensor 23 detects the speed of the vehicle 200 and generates speed information indicating the detected speed.

[0073] The information acquisition device 20 outputs the position information, the vehicle exterior information, and the speed information to the display processing device 30. In addition, the image data generated by the camera 22 may also be output to the display processing device 30.

[0074] The display processing device 30 includes a communication unit 31 , a display control unit 32 , and a storage unit 33 .

[0075] The communication unit 31 includes a circuit for communicating with an external device in accordance with a predetermined communication standard, such as LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), USB, HDMI (registered trademark), CAN (controller area network), and SPI (Serial Peripheral Interface).

[0076] The display control unit 32 can be implemented by semiconductor elements and the like. The display control unit 32 can include, for example, a microcomputer, a CPU, an MPU, a GPU, a DSP, an FPGA, or an ASIC. The functions of the display control unit 32 can be configured only by hardware or by combining hardware and software. The display control unit 32 realizes the prescribed functions by reading the data and programs stored in the storage unit 33 and performing various calculations.

[0077] The storage unit 33 is a storage medium that stores programs and data required to realize the functions of the display processing device 30. The storage unit 33 can be realized by, for example, a hard disk (HDD), SSD, RAM, DRAM, ferroelectric memory, flash memory, magnetic disk, or a combination thereof.

[0078] The storage unit 33 stores a plurality of image data 330 representing the virtual image Iv.

[0079] The display control unit 32 determines the virtual image Iv to be displayed based on the position information and vehicle exterior information obtained from the information acquisition device 20. The display control unit 32 reads the image data 330 of the determined virtual image Iv from the storage unit 33 and outputs it to the projection device 10. The display control unit 32 obtains information indicating the reference position of the display of the virtual image Iv from an external device (not shown) via the communication unit 31. The display control unit 32 outputs the speed information indicating the vehicle speed obtained from the information acquisition device 20 to the correction processing device 50, and obtains the correction amount of the display position from the correction processing device 50. The display control unit 32 sets the display position of the virtual image Iv based on the reference position and the correction amount.

[0080] The posture detection device 40 includes a gyro sensor 41 that detects an angular velocity. The gyro sensor 41 outputs angular velocity information indicating the detected angular velocity to the correction processing device 50.

[0081] The correction processing device 50 includes a communication unit 51 , a correction control unit 52 , and a storage unit 53 .

[0082] The communication unit 51 includes a circuit for communicating with an external device in accordance with a predetermined communication standard, such as LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), USB, HDMI (registered trademark), CAN (controller area network), and SPI (Serial Peripheral Interface).

[0083] The correction control unit 52 can be implemented by a semiconductor element or the like. The correction control unit 52 can include, for example, a microcomputer, a CPU, an MPU, a GPU, a DSP, an FPGA, or an ASIC. The functions of the correction control unit 52 can be implemented by hardware alone or by combining hardware with software. The correction control unit 52 performs various calculations by reading data and programs stored in the storage unit 53, thereby implementing a predetermined function.

[0084] The storage unit 53 is a storage medium that stores programs and data required to realize the functions of the correction processing device 50. The storage unit 53 can be realized by, for example, a hard disk (HDD), SSD, RAM, DRAM, ferroelectric memory, flash memory, magnetic disk, or a combination of these.

[0085] The storage unit 53 stores threshold value data 530 indicating a threshold value according to the vehicle speed.

[0086] The correction control unit 52 includes, as a functional configuration, a threshold setting unit 52a, a determination unit 52b, a deviation amount calculation unit 52c, and a correction amount calculation unit 52d.

[0087] The threshold setting unit 52 a sets a first threshold a which is a threshold of the angular velocity based on the speed information of the vehicle 200 and the threshold data 530 .

[0088] The determination unit 52b compares the angular velocity of the vehicle 200 with the first threshold value a set by the threshold setting unit 52a, and outputs the comparison result. If the determination unit 52b determines that the magnitude of the angular velocity is greater than the first threshold value a, it can be inferred that the correction amount is greater than the prescribed amount that exceeds the correctable range. In addition, if the determination unit 52b determines that the magnitude of the angular velocity is less than the first threshold value a, it can be inferred that the correction amount is less than the prescribed amount within the correctable range.

[0089] The deviation amount calculation unit 52c calculates the posture (angle deviation) of the vehicle 200 based on the posture change information output by the posture detection device 40. For example, the deviation amount calculation unit 52c calculates the angle (pitch angle) of the vehicle 200 around the pitch axis by integrating the angular velocity detected by the gyro sensor 41. Figure 1 The deviation (angle) of the vehicle 200 in the rotation direction around the Y axis (pitch axis) is shown. Similarly, the yaw angle or the roll angle can also be calculated, for example, by calculating all the angles around the X axis, the Y axis, and the Z axis. Figure 1 The deviation of the vehicle 200 in the X-axis, Y-axis, and Z-axis, i.e., the rolling direction, the pitch direction, and the yaw direction, is shown in FIG. In addition, in the present embodiment, all angles around the three axes are calculated, but angles around one or two axes may also be calculated. For example, only the angles around the Y-axis and the Z-axis may be calculated.

[0090] The correction amount calculation unit 52d calculates the correction amount of the display position of the virtual image Iv according to the posture (angle deviation) of the vehicle 200. The correction amount is represented by the number of pixels, for example. Specifically, the correction amount calculation unit 52d converts the deviation of the pitch angle and the yaw angle calculated by the deviation amount calculation unit 52c from the angle into the number of pixels, and determines the correction amount for returning the number of pixels that have deviated to the original. For example, with respect to the roll angle, the correction amount calculation unit 52d determines the correction amount for returning the deviation of the roll angle to the original under the state of the angle. The correction amount calculation unit 52d outputs the calculated correction amount to the display processing device 30.

[0091] As described above, the display processing device 30 and the correction processing device 50 communicate bidirectionally via the communication units 31 and 51. The display processing device 30 outputs speed information indicating the vehicle speed to the correction processing device 50. The correction processing device 50 outputs information indicating the correction amount to the display processing device 30.

[0092] Figure 3An example of the threshold value data 530 is shown. The threshold value data 530 is a table showing the first threshold value a corresponding to the vehicle speed V. Figure 4A The first threshold value a is a graph showing the threshold value data 530. The first threshold value a is set to increase in stages based on the vehicle speed V so that the faster the vehicle speed V, the larger the first threshold value a. The threshold value setting unit 52a compares the speed information of the vehicle 200 with the threshold value data 530 and sets the first threshold value a corresponding to the speed V of the vehicle 200. In addition, the first threshold value a may be set by a function depending on the vehicle speed V. For example, Figure 4B As shown in the graph, it can be represented by a linear function (a=k×V+b). Here, k represents the slope and b represents the intercept. In addition, it can also be represented by a function of quadratic or higher, an exponential function, etc. In addition, different functions can be set for each speed V range.

[0093] 2. AR Display

[0094] Reference Figures 5 to 10 , explaining the AR display.

[0095] Figure 5 An example of a real scene viewed from a windshield 210 of a vehicle 200 is shown. Figure 6 FIG. 2 is an example of a virtual image Iv viewed from the display area 220. The display system 100 uses Figure 5 The real scene shown Figure 6 The virtual image Iv shown is overlapped. The reference position P0 of the virtual image Iv is a position determined based on the type of virtual image Iv, the state of the vehicle 200, such as the position and posture of the vehicle 200, and map data, and the reference position P0 is determined by an external device. For example, when the display object 230 is a driving lane and the virtual image Iv is an arrow indicating the direction of travel, the display position of the arrow when the arrow indicates the center of the driving lane when the vehicle is stationary is the reference position P0. The reference position P0 is, for example, Figure 6The reference position P0 is set by the position of the pixel on the liquid crystal display corresponding to the values ​​of the Y coordinate and the Z coordinate in the display area 220. The reference position P0 is obtained from an external device. The external device includes, for example, a microcomputer, a CPU, an MPU, a GPU, a DSP, an FPGA, or an ASIC, a GPS module 21. The function of the external device can be constituted by hardware alone, or it can be realized by combining hardware and software. The information representing the reference position P0 output from the external device may change based on the change of the posture caused by the number of passengers, the change of the load, and the reduction of gasoline. Therefore, for example, the reference position P0 obtained from the external device may be different from the initial position obtained initially. Therefore, the display processing device 30 may also change the reference position P0 obtained from the external device based on the change of the posture caused by the number of passengers, the change of the load, and the reduction of gasoline. In addition, the display processing device 30 may also set the reference position P0 based on the position information, the information outside the vehicle, and the map data. The display processing device 30 may also set the size of the virtual image Iv based on the position information and the information outside the vehicle.

[0096] Fig. 7A The vehicle 200 is shown without being tilted. Figure 7B A display example of the virtual image Iv when the vehicle 200 is not tilted is shown. Figure 7B express Figure 5 The real scene shown Figure 6 When the vehicle 200 is not tilted, as shown in FIG. Figure 7B As shown in FIG. 1 , if the virtual image Iv is displayed at the reference position P0 , the virtual image Iv is displayed at a desired position to be displayed, for example, the center of the driving lane.

[0097] Fig. 8A The vehicle 200 is shown in a forward leaning posture. Figure 8B A display example of the virtual image Iv when the vehicle 200 is in a forward leaning posture is shown. Figure 8B The example shows that the display position of the virtual image Iv deviates from the display object 230 according to the change of the posture of the vehicle 200. The vehicle 200 may tilt due to unevenness of the road surface, sudden acceleration or sudden deceleration of the vehicle 200, etc. For example, if the vehicle 200 suddenly decelerates, Fig. 8A The vehicle 200 is shown in a forward leaning posture. Figure 8B As shown in FIG. 1 , the position of the display object 230 viewed from the windshield 210 changes according to the inclination of the vehicle 200. Therefore, when the virtual image Iv is displayed at the reference position P0, the virtual image Iv deviates from the display object 230. Figure 8B As shown, the tip of the arrow is in the oncoming lane 231. Therefore, the display system 100 adjusts the display position of the virtual image Iv in the direction to restore the deviation corresponding to the posture of the vehicle 200.

[0098] Fig. 9 Indicates the display position of the virtual image Iv before and after correction. The correction processing device 50 calculates the correction amount c so that the display position of the virtual image Iv becomes the position P1 without deviation caused by the angle of the vehicle 200. That is, the display processing device 30 sets the display position of the virtual image Iv to "reference position P0+correction amount c". Thus, the projection device 10 can display the virtual image Iv at the position P1 desired to be displayed for the display object 230. In this way, even if the vehicle 200 is tilted, the display position of the virtual image Iv can be changed from the reference position P0 based on the correction amount c, so that the virtual image Iv can be displayed at the position P1 desired to be displayed for the display object 230 in the real scene.

[0099] Fig.10 An example is given of a case where the display position of the virtual image Iv deviates from the display object 230 due to the noise of the gyro sensor 41. As described above, for example, the angular velocity detected by the gyro sensor 41 includes an error due to drift, so if the correction amount c is calculated based on the integral operation of the angular velocity, the correction amount c includes an error. In this case, for example, even when the vehicle 200 is stopped and there is almost no vibration, the posture change of the vehicle 200 can be detected, and the correction amount c is not zero. Therefore, the display position (=reference position P0+correction amount c) of the virtual image Iv changes, for example, deviates from the position P2 where it is desired to be displayed. In order to reduce the position deviation E caused by the noise of the sensor, the present embodiment resets the correction amount c to zero when the angular velocity is greater than the first threshold a, as described later. In this way, it is possible to suppress the accumulation of the correction error caused by the drift of the gyro sensor 41.

[0100] 3. Display processing device operation

[0101] Reference Fig.11 , the operation of the display control unit 32 of the display processing device 30 is described. Fig.11 The display processing performed by the display control unit 32 of the display processing device 30 is shown. Fig.11 The display processing shown is started, for example, when the engine of the vehicle 200 is started, or when a button for instructing the start of display of the virtual image Iv is operated.

[0102] The display control unit 32 acquires the position information, vehicle exterior information, and speed information of the vehicle 200 from the information acquisition device 20 (S101). The display control unit 32 outputs the speed information to the correction processing device 50 (S102). The display control unit 32 determines whether to display a virtual image Iv corresponding to the display object based on the position information and vehicle exterior information (S103).

[0103] When the display control unit 32 determines to display the virtual image Iv (Yes in S104), it obtains information indicating the reference position P0 of the virtual image Iv from the external device (S105). The display control unit 32 obtains information indicating the correction amount c of the display position output from the correction processing device 50 (S106). The display control unit 32 causes the projection device 10 to display the virtual image Iv based on the reference position P0 and the correction amount c (S107). For example, the display control unit 32 reads the image data 330 of the virtual image Iv corresponding to the display object from the storage unit 33, sets the display position of the virtual image Iv to "reference position P0+correction amount c", and outputs the image data 330 and the information indicating the display position to the projection device 10.

[0104] When the display control unit 32 determines not to display the virtual image Iv (No in S104), the display control unit 32 sets the virtual image Iv to be non-displayed (S108). For example, the display control unit 32 outputs a command to cause the projection device 10 to stop displaying the virtual image Iv.

[0105] The display control unit 32 determines whether to continue the display process (S109). For example, when the engine of the vehicle 200 stops, or when a button for indicating the end of the display of the virtual image Iv is operated, the display control unit 32 ends the display process. In this case, the display control unit 32 stops the display of the virtual image Iv of the projection device 10. In the case of continuing the display process, the process returns to step S101.

[0106] 4. Correction of the processing device's actions

[0107] Reference Fig.12 as well as Fig.13A , the operation of the correction control unit 52 of the correction processing device 50 in the first embodiment will be described. Fig.12 The correction processing performed by the correction control unit 52 of the correction processing device 50 is shown. Fig.13A The functional configuration of the correction control unit 52 in the first embodiment is shown.

[0108] Fig.12 The correction process shown is started, for example, when the engine of the vehicle 200 is started, or when a button for instructing the start of display of the virtual image Iv is operated. Fig.12 The correction processing is as follows Fig.11 In addition, Fig.12 The correction processing shown may be started when a button for instructing the start of the position correction of the virtual image Iv is operated.

[0109] The correction control unit 52 obtains angular velocity information indicating the angular velocity of the vehicle 200 output from the gyro sensor 41 (S201). The deviation amount calculation unit 52c calculates the deviation amount of the vehicle 200 relative to the three axes based on the angular velocity (S202). Fig.13A As shown in FIG. 1 , the deviation calculation unit 52c calculates the current deviation y by “y=y′+x”. The deviation y is the angle around the three axes. Fig.13A In the equation, y' is the previous deviation, and x is the calculated value of the integration operation process. The calculated value x is calculated by "x = (gyro_in + gyro_in') × K". K is a filter coefficient. gyro_in is the angular velocity obtained in step S201, and gyro_in' is the previous angular velocity.

[0110] The threshold setting unit 52a obtains speed information indicating the vehicle speed from the display processing device 30 (S203). The threshold setting unit 52a sets a first threshold a corresponding to the vehicle speed based on the threshold data 530 stored in the storage unit 53 and the speed information obtained in step S203 (S204). The determination unit 52b determines whether the angular velocity obtained in step S201 is less than or equal to the first threshold a (S205).

[0111] When the determination unit 52b determines that the angular velocity is less than the first threshold value a (Yes in S205), the correction amount calculation unit 52d calculates the correction amount c of the display position of the virtual image Iv based on the current deviation amount y (S206). When the determination unit 52b determines that the angular velocity is greater than the first threshold value a (No in S205), the correction amount calculation unit 52d resets the correction amount c to zero (S207).

[0112] For example, in step S206, Fig.13A As shown, the correction amount calculation unit 52d calculates the correction amount c by "c = (y-ofs) × G". Here, G is a conversion factor for converting an angle into a number of pixels. Specifically, for example, the correction amount calculation unit 52d converts the angle of the vehicle 200, that is, the deviation amount, into a number of pixels for the pitch angle and the yaw angle, and determines the correction amount c that cancels the deviation amount indicated by the number of pixels. Regarding the roll angle, the correction amount c that cancels the deviation amount is determined in the state of the angle. The initial value of the offset value ofs is, for example, zero.

[0113] For example, when the correction amount c is reset to zero in step S207, the correction amount calculation unit 52d sets the current deviation y to the offset value ofs (ofs = y). That is, the offset value ofs is the deviation y when the angular velocity is greater than the first threshold a. As a result, the calculation of the correction amount c in the correction amount calculation unit 52d, "c = (y-ofs) × G", becomes "c = 0 × G" through "ofs = y". Therefore, the correction amount c calculated by the correction amount calculation unit 52d is zero.

[0114] The correction amount calculation unit 52d outputs the correction amount c calculated in step S206 or the correction amount c calculated in step S207 to the display processing device 30 (S208).

[0115] The correction control unit 52 determines whether to continue the correction process (S209). If the correction process is to be continued (Yes in S209), the process returns to step S201. If the correction process is not to be continued (No in S209), the process ends. Fig.12 Processing shown.

[0116] 5. Effects and supplements, etc.

[0117] The display system 100 disclosed in the present invention includes an information acquisition device 20, a posture detection device 40, a display processing device 30, and a correction processing device 50. The information acquisition device 20 acquires speed information indicating the speed of the moving body. The posture detection device 40 detects the posture change amount of the moving body. The display processing device 30 controls the display position of the image based on the reference position and the correction amount. The correction processing device 50 sets the correction amount c based on the speed and the posture change amount. Specifically, the correction processing device 50 sets the first threshold a based on the speed. The correction processing device 50 sets the correction amount c based on the posture change amount during a period when the posture change amount is less than the first threshold a. The display processing device 30 returns the display position to the reference position P0 during a period when the posture change amount is greater than the first threshold a.

[0118] Specifically, the posture change amount is the angular velocity of the moving object. The display system 100 of this embodiment further includes a projection device 10 that projects light representing an image. In this embodiment, the moving object is a vehicle 200, and the image is a virtual image displayed in front of the windshield of the vehicle 200.

[0119] The threshold setting unit 52a of the correction processing device 50 changes the size of the first threshold a according to the vehicle speed, so that the position deviation of the image can be suppressed with high accuracy. Specifically, the threshold setting unit 52a sets the first threshold a so that Figure 4A as well as Figure 4BIn this way, the faster the vehicle speed, the larger the first threshold value a. Thus, the vibration amplitude (deviation) can be determined (inferred) based on the vehicle speed and the amount of posture change, and when the vibration amplitude is within the correction range, the position deviation correction is implemented. When the vibration amplitude exceeds the correction range and the position deviation correction cannot be performed, when the change of the external scene is large, a reset is performed to eliminate the cumulative error of the correction value. Thus, it is possible to perform a position deviation correction of the image with excellent accuracy while maintaining the accuracy of the position deviation correction, taking into account both the elimination of the cumulative error and the quality of the appearance. For example, even when the vehicle 200 is traveling at a low speed and the vibration frequency is low, the display position can be returned to the reference position. In addition, even when the vehicle 200 is traveling at a high speed and the vibration frequency is high, the display position can be changed with a correction amount based on the angular velocity. By changing the display position based on the angular velocity, the position deviation of the image based on the posture change of the moving body can be suppressed with high accuracy. In addition, by returning the display position to the reference position, the accumulation of correction errors can be reduced, and the position deviation of the image caused by the accumulation of correction errors can be suppressed. According to the present embodiment, the first threshold value a is set according to the vehicle speed, so that the correction based on the angular velocity and the reset of the correction error can be accurately taken into account according to the driving speed. Even if the display position of the image is returned to the reference position when the deviation amount of the display position of the image becomes large, the observer will not feel uncomfortable with the change of the display position of the image.

[0120] Specifically, the correction processing device 50 resets the correction amount c to zero when the angular velocity is greater than the first threshold a. More specifically, the correction processing device 50 includes a deviation amount calculation unit 52c and a correction amount calculation unit 52d. The deviation amount calculation unit 52c is equivalent to a first calculation unit that calculates the angle of the moving body based on the angular velocity. The correction amount calculation unit 52d is equivalent to a second calculation unit that converts the angle into the number of pixels to calculate the correction amount c.

[0121] For example, since the angular velocity detected by the gyro sensor 41 includes an error due to drift, if the calculation of the correction amount c based on the integral operation of the angular velocity is continued, the error included in the correction amount c is accumulated and becomes larger. However, in this embodiment, the correction amount c is reset to zero by setting the deviation amount y when the angular velocity is greater than the first threshold a as the offset value ofs. Thus, when the angular velocity is greater than the first threshold a, the display position of the virtual image Iv is reset to the reference position P0. Therefore, when the angular velocity is greater than the first threshold a, the error in the display position caused by the accumulation of noise of the gyro sensor 41 used to detect the vehicle posture can be eliminated. Thus, the display position of the virtual image Iv can be restored to the position where it is desired to be displayed. The correction amount c when the angular velocity is less than the first threshold a is calculated by "c = (y-ofs) × G". By setting the offset value ofs to the deviation amount y when the angular velocity is greater than the first threshold a, the accumulation of the correction error caused by the noise of the gyro sensor 41 can be suppressed.

[0122] exist Fig.13A In the example, the offset value ofs is an angle, but the offset value ofs can also be a number of pixels. Fig. 13B Another example of the functional structure of the correction control unit 52 in the first embodiment is shown. Fig. 13B As shown, the correction amount calculation unit 52d calculates the correction amount c by "c = y × G - ofs". In this case, the correction amount calculation unit 52d sets "y × G" when the angular velocity is greater than the first threshold a as the offset value ofs (ofs = y × G). Thus, the correction amount c when the angular velocity is greater than the first threshold a can also be reset to zero.

[0123] (Second embodiment)

[0124] In the first embodiment, the correction amount calculation unit 52d of the correction processing device 50 outputs the correction amount c adjusted according to the offset value ofs, and the display processing device 30 sets the display position of the virtual image Iv to "reference position P0 + correction amount c". In the present embodiment, the correction amount calculation unit 52d outputs the correction amount c and the offset value ofs separately. That is, in the present embodiment, the correction amount c is not adjusted according to the offset value ofs. The display processing device 30 sets the display position of the virtual image Iv to "reference position P0 + offset value ofs + correction amount c".

[0125] Fig.14 FIG. 2 shows the display processing performed by the display control unit 32 of the display processing device 30. Fig.14 Steps S301 to S305, S308, and S309 are the same as those of the first embodiment. Fig.11 Steps S101 to S105, S108, and S109 are the same.

[0126] In the present embodiment, the display control unit 32 acquires the offset value ofs from the correction processing device 50 together with the correction amount c when displaying the virtual image (S306). The display control unit 32 causes the projection device 10 to display the virtual image Iv (S307) based on the reference position P0, the offset value ofs and the correction amount c. Specifically, the display control unit 32 sets a new reference position P0' by "P0'=P0+ofs" according to the reference position P0 and the offset value ofs. The reference position P0 before adjustment by the offset value ofs is also referred to as the initial position. The offset value ofs is equivalent to the offset from the initial position. The display control unit 32 sets the display position of the virtual image Iv to "new reference position P0'+correction amount c" and causes the projection device 10 to display the virtual image Iv.

[0127] Reference Fig.15 as well as Fig.16 , the operation of the correction control unit 52 of the correction processing device 50 in the second embodiment will be described. Fig.15 FIG. 2 shows the correction process performed by the correction control unit 52 of the correction processing device 50. Fig.15 Steps S401, S402, S404 to S406, and S409 are respectively the same as those of the first embodiment. Fig.12 Steps S201 to S205 and S209 are the same. Fig.16 The functional configuration of the correction control unit 52 in the second embodiment is shown.

[0128] In the present embodiment, the correction amount calculation unit 52d calculates the correction amount c based on the deviation amount y by “c=y×G” (S403).

[0129] The correction amount calculation unit 52d sets an offset value ofs (ofs=-c) based on the correction amount c when the angular velocity is greater than the first threshold a (S407). The offset value ofs in this embodiment corresponds to the number of pixels. The initial value of the offset value ofs is, for example, zero.

[0130] The correction amount calculation unit 52d outputs the correction amount c calculated in step S403 and the offset value ofs set in step S407 to the display processing device 30 (S408).

[0131] As described above, in this embodiment, the display processing device 30 controls the display position of the image based on the reference position P0, the offset value ofs and the correction amount c. The correction processing device 50 sets the offset value ofs based on the correction amount c when the angular velocity is greater than the first threshold a.

[0132] By setting the offset value ofs based on the correction amount c when the angular velocity is greater than the first threshold a, the display position based on the new reference position P0' (= P0 + ofs) and the correction amount c of the display control unit 32 is substantially the same as the display position of the first embodiment. Therefore, the present embodiment can achieve the same effects as the first embodiment.

[0133] (Third embodiment)

[0134] In the first embodiment, the correction control unit 52 resets the correction amount c to zero when the angular velocity is greater than the first threshold a. In the present embodiment, the correction control unit 52 decreases the magnitude of the correction amount c by a predetermined amount when the angular velocity is greater than the first threshold a.

[0135] Fig.17 FIG. 2 shows the correction processing performed by the correction control unit 52 of the correction processing device 50 in the third embodiment. Fig.17 Steps S501, S502, S504 to S506, S509, and S510 are respectively the same as those of the first embodiment. Fig.12 Steps S201 to S205, S208, and S209 are the same.

[0136] The correction control unit 52 calculates the correction amount c of the display position of the virtual image Iv based on the deviation amount y, for example, by "c=y×G" (S503). When the determination unit 52b determines that the angular velocity is less than the first threshold a (Yes in S506), the correction amount calculation unit 52d outputs the correction amount c calculated in step S503 (S509).

[0137] When the determination unit 52b determines that the angular velocity is greater than the first threshold a (No in S506), the correction amount calculation unit 52d determines whether the correction amount c calculated in step S503 is zero (S507). Alternatively, the determination unit 52b may determine whether the correction amount c is zero.

[0138] If the correction amount c is not zero (No in step S507), the correction amount calculation unit 52d reduces the magnitude of the correction amount c by a predetermined amount so that the correction amount c approaches zero (S508). For example, the correction amount calculation unit 52d subtracts the predetermined amount q from the correction amount c calculated in step S503. px , in step S509, output "cq px In another example, the correction amount calculation unit 52d may subtract a predetermined amount q from the deviation amount y. deg , through "c=(yq deg )×G” to calculate the correction amount c. In another example, the correction amount calculation unit 52d may also be Fig.13A In the calculation of the correction amount c shown in FIG. 1 , the prescribed amount q degThe correction amount calculation unit 52d may also be configured to Fig. 13B In the calculation of the correction amount c shown in FIG. 1 , the prescribed amount q px Set to the offset value ofs. px ,q deg It can also be set according to the size of the deviation amount y or the correction amount c. px is set to a value smaller than the correction amount c so that the correction amount c becomes a value greater than zero. px ,q deg The size of may also be set according to the display position of the virtual image Iv in the display area 220. If the correction amount c is zero (Yes in step S507), step S508 is not executed and the process proceeds to step S509.

[0139] The correction amount calculation unit 52d outputs the correction amount c calculated in step S503 or the correction amount c calculated in step S508 to the display processing device 30 (S509).

[0140] As described above, the correction processing device 50 calculates the correction amount c according to each sampling period of the correction processing, and when the angular velocity is greater than the first threshold value a, the correction amount c is reduced by a predetermined amount. In this way, by updating the correction amount and reducing the correction amount by a certain amount, the position of the virtual image Iv does not change sharply or significantly. Therefore, it is possible to suppress the passenger D from feeling uncomfortable with the change in the display position of the virtual image Iv. In other words, it is possible to suppress the uncomfortable feeling of the appearance caused by the offset of the display position. Furthermore, it is also possible to suppress the cumulative error.

[0141] (Fourth embodiment)

[0142] When the angular velocity is greater than the first threshold a, the correction amount c is reset to zero in the first embodiment, and the magnitude of the correction amount c is reduced by a predetermined amount in the third embodiment. In this embodiment, the correction amount c is adjusted according to the magnitude of the correction amount c calculated using the deviation. Specifically, when the correction amount c is greater than or equal to the second threshold b, the correction amount c is reduced by a predetermined amount, and when the correction amount c is less than the second threshold b, the correction amount c is reset to zero. The second threshold b is a threshold of the correction amount c.

[0143] Fig.18 The correction process in the fourth embodiment is shown. Fig.18 The correction process shown is to convert the first embodiment Fig.12 The correction process shown and the third embodiment Fig.17 The correction treatment combination shown.

[0144] In this embodiment, when the angular velocity is greater than the first threshold a (No in S506), the correction amount calculation unit 52d determines whether the correction amount c based on the deviation amount y calculated in step S503 is greater than or equal to the second threshold b (S517). The determination in step S517 may also be performed by the determination unit 52b.

[0145] If the correction amount c is equal to or greater than the second threshold value b (Yes in step S517 ), the correction amount calculation unit 52 d decreases the correction amount c by a predetermined amount ( S508 ).

[0146] If the correction amount c is smaller than the second threshold value b (No in step S517), the correction amount calculation unit 52d resets the correction amount c to zero (S518).

[0147] As described above, the correction processing device 50 calculates the correction amount c at each sampling period of the correction processing, and when the angular velocity is greater than the first threshold value a and the correction amount c is greater than the second threshold value b, the correction amount c is reduced by a predetermined amount so that the correction amount c is close to zero, and when the correction amount c is less than the second threshold value b, the correction amount c is reset to zero. In this way, if the correction amount c is updated and reduced by a certain amount, and reset when it becomes smaller to a certain extent, it is possible to correct the display position and eliminate the accumulated error without causing any discomfort to the appearance according to the inclination of the posture of the vehicle 200.

[0148] (Fifth embodiment)

[0149] In this embodiment, the method of resetting the correction amount c to zero is different from that in the first embodiment. Fig.13A as well as Fig. 13B In the embodiment, the correction amount calculation unit 52d resets the correction amount c to zero. In the present embodiment, the deviation amount calculation unit 52c resets the deviation amount y to zero, thereby resetting the correction amount c to zero.

[0150] Fig.19 1 is a block diagram showing the internal structure of the display system 100 according to the fifth embodiment. In the present embodiment, the determination result of the determination unit 52b is output to the deviation amount calculation unit 52c.

[0151] Fig. 20 FIG. 5 shows the correction process performed by the correction control unit 52 of the fifth embodiment. Fig. 20 Steps S601 to S605, S608, and S609 are respectively the same as those of the first embodiment. Fig.12 Steps S201, S203~S205, S202, S208, and S209 are the same. Fig.21A The functional structure of the correction control unit 52 according to the fifth embodiment is shown.

[0152] When the determination unit 52b determines that the angular velocity is less than the first threshold value a (Yes in S604), the deviation amount calculation unit 52c calculates the deviation amount y based on the angular velocity by "y=y'+x" (S605). In the present embodiment, if the determination unit 52b determines that the angular velocity is greater than the first threshold value a (No in S604), the deviation amount calculation unit 52c resets the deviation amount y to zero (S606).

[0153] For example, Fig.21A As shown, the deviation calculation unit 52c sets x=0 and y'=0 and outputs "y=0". By setting "x=0" and "y'=0", the integration filter in the deviation calculation unit 52c is reset. As a result, the accumulation of the error in the integral operation in the deviation calculation unit 52c is eliminated.

[0154] The correction amount calculation unit 52d calculates the correction amount c (S607) by "c=y×G" based on the deviation amount calculated in step S605 or step S606. When the angular velocity is greater than the first threshold a, the deviation amount calculation unit 52c outputs zero, so the correction amount calculation unit 52d calculates "c=0×G". As a result, the correction amount c is reset to zero.

[0155] In addition, the offset can be reset to zero by other methods. Fig. 21B Another example of the functional configuration of the correction control unit 52 according to the fifth embodiment is shown. In this example, the deviation amount calculation unit 52c outputs the difference "y-ofs" between the deviation amount y and the offset value ofs to the correction amount calculation unit 52d. Fig. 21B The offset value ofs is Fig.13A The deviation y when the angular velocity is greater than the first threshold a is set to the offset value ofs (ofs=y). Thus, the correction amount c when the angular velocity is greater than the first threshold a is reset to zero. By setting the deviation y when the angular velocity is greater than the first threshold a to the offset value ofs, the accumulation of errors in the integral operation in the deviation amount calculation unit 52c can be eliminated.

[0156] As described above, in this embodiment, when the angular velocity is greater than the first threshold value a, the deviation amount calculation unit 52c resets the deviation amount to zero. Thus, the correction amount c is reset to zero. According to this embodiment, the same effect as the first embodiment can be obtained.

[0157] (Sixth embodiment)

[0158] When the angular velocity is larger than the first threshold a, the correction amount c is reduced by a predetermined amount in the third embodiment. In the present embodiment, while the angular velocity is larger than the first threshold a, the correction amount c is gradually reset to zero over a certain period of time.

[0159] Fig. 22The correction process in the sixth embodiment is shown. Fig. 22 Steps S701 to S707, S711, and S713 are respectively the same as those of the third embodiment. Fig.17 Steps S501 to S507, S509, and S510 are the same.

[0160] The correction control unit 52 calculates the correction amount c of the display position of the virtual image Iv based on the deviation amount y, for example, by "c=y×G" (S703). When the determination unit 52b determines that the angular velocity is less than the first threshold a (Yes in S706), the correction amount calculation unit 52d outputs the correction amount c calculated in step S703 (S711).

[0161] When the determination unit 52b determines that the angular velocity is greater than the first threshold a (No in S706), the correction amount calculation unit 52d determines whether the correction amount c calculated in step S703 is zero (S707). Alternatively, the determination unit 52b may determine whether the correction amount c is zero.

[0162] When the correction amount calculation unit 52d determines that the correction amount c is not zero (No in step S707), the correction amount calculation unit 52d determines whether the reset start flag is set to ON (S708). If the correction amount calculation unit 52d determines that the reset start flag is not set to ON (No in S708), the correction amount calculation unit 52d sets the reset start flag to ON and calculates the second offset ofs2 (S709). Next, the correction amount calculation unit 52d reduces the correction amount c by the calculated second offset ofs2 (S710). The correction amount calculation unit 52d outputs the reduced correction amount c to the display processing device 30 (S711).

[0163] Next, the process returns to step S707, and the correction amount calculation unit 52d determines whether the correction amount c is zero again. If the correction amount calculation unit 52d determines that the correction amount c is not zero (No in S707), the correction amount calculation unit 52d determines whether the reset start flag is set to ON. If the reset start flag is set to ON (Yes in S708), the correction amount c is reduced by the offset amount ofs2 again (S710). In this way, the correction amount c is gradually reduced.

[0164] For example, if the reset start flag is set to ON at time t1, the correction amount C gradually decreases while the reset start flag is set to ON, and at time t4 after the reset time Δt1 from time t1, the correction amount becomes 0. Alternatively, the reset time Δt1 may be set in advance, and the second offset ofs2 in one sampling (one cycle from S707 to S711 in the flowchart) may be set to c1×ts / Δt1 based on the sampling cycle ts and the correction amount c1 at the start of the reset, and the correction amount may be reduced by c1×ts / Δt1 each time.

[0165] If the correction amount c becomes zero, the correction amount calculation unit 52d determines that the correction amount c is zero in the judgment in step S707 (Yes in S707), and sets the reset start flag to OFF (S712). The correction amount calculation unit 52d outputs the correction amount c calculated in step S703 or the correction amount c repeatedly reduced and becoming zero in steps S704 to S710 to the display processing device 30 (S711).

[0166] As described above, the correction processing device 50 gradually decreases the correction amount c over a certain period of time when the angular velocity is greater than the first threshold a, so that the position of the virtual image Iv gradually returns to the initial position P0. Since the position of the virtual image Iv does not change suddenly and greatly, it is possible to suppress the passenger D from feeling uncomfortable with the change in the display position of the virtual image Iv. In other words, it is possible to suppress the uncomfortable feeling of the appearance caused by the deviation of the display position.

[0167] (Seventh Implementation Method)

[0168] When the angular velocity is greater than the first threshold a, when the correction amount is greater than the second threshold b, in the fourth embodiment, the correction amount calculation unit 52d reduces the correction amount c by a predetermined amount. In this embodiment, when the correction amount c is greater than the second threshold b, the correction amount c is gradually reduced, and when the correction amount c is less than the second threshold b, the correction amount is reset to zero.

[0169] Fig.23 The correction process in the seventh embodiment is shown. Fig.23 Steps S801 to S807, S811, and S813 are respectively the same as those of the fourth embodiment. Fig.18 Steps S501 to S517, S509, and S510 are the same as those of the seventh embodiment. Fig.23 Steps S808 to S810 of the sixth embodiment are respectively Fig. 22 The steps S708 to S710 are the same.

[0170] In this embodiment, when the angular velocity is greater than the first threshold a (No in S806), the correction amount calculation unit 52d determines whether the correction amount c based on the deviation y calculated in step S803 is greater than or equal to the second threshold b (S807). The determination in step S807 may also be performed by the determination unit 52b.

[0171] If the correction amount c is greater than or equal to the second threshold value b (Yes in step S807), the correction amount calculation unit 52d determines whether the reset start flag is set to ON (S808). If the correction amount calculation unit 52d determines that the reset start flag is not set to ON (No in S808), the correction amount calculation unit 52d sets the reset start flag to ON and calculates the second offset ofs2 (S809). Next, the correction amount calculation unit 52d reduces the correction amount c by the calculated second offset ofs2 (S810). The correction amount calculation unit 52d outputs the reduced correction amount c to the display processing device 30 (S711).

[0172] Next, the process returns to step 807, and the correction amount calculation unit 52d again determines whether the correction amount c is zero. If the correction amount calculation unit 52d determines that the correction amount c is greater than the second threshold value b (yes in S807), the correction amount calculation unit 52d determines whether the reset start flag is set to ON (S808). If the reset start flag is set to ON (yes in S808), the correction amount c is reduced by the offset ofs2 again (S810). In this way, the correction amount c is gradually reduced. If the correction amount c is less than the second threshold value b, the correction amount calculation unit 52d determines in the judgment in step S807 that the correction amount c is less than the second threshold value b (no in S807), and the correction amount calculation unit 52d resets the correction amount c to zero (S814). Then, the correction amount calculation unit 52d sets the reset start flag to OFF (S812).

[0173] For example, if the reset start flag is set to ON at time t1, the correction amount gradually decreases during the period when the reset start flag is set to ON, and the correction amount becomes less than the second threshold value b at time t5 after the reset time Δt2 from time t1. Alternatively, the reset time Δt2 may be set in advance, and the offset in one sampling (one cycle to S807 to S811 in the flowchart) may be set to (C1-b)×ts / Δt2 based on the sampling cycle ts and the correction amount c1 at the start of the reset, and the correction amount may be reduced by (C1-b)×ts / Δt2 each time. If the correction amount c is less than the second threshold value b, the correction amount c is immediately reset to zero.

[0174] As described above, when the angular velocity is greater than the first threshold value a and the correction amount c is greater than the second threshold value b, the correction processing device 50 reduces the correction amount c by a certain amount each time so that the correction amount c approaches zero, and when the correction amount c is less than the second threshold value b, the correction amount c is reset to zero. Thus, the display position can be corrected and the cumulative error can be eliminated without causing any discomfort to the appearance.

[0175] (Eighth Implementation Method)

[0176] The first to seventh embodiments describe a display system 100 that displays a virtual image in front of a vehicle windshield. However, correction of the image display position according to the present disclosure is not limited to the display system 100 having a plurality of devices, and may be implemented by a single device.

[0177] Fig.24 The structure of the display device in the eighth embodiment is shown. The display device 600 of this embodiment is, for example, a device that displays images according to the travel of the vehicle 200. The display device 600 is, for example, a personal computer, a tablet terminal, a smart phone, or other information processing device. The display device 600 is, for example, equivalent to Figure 2 The display processing device 30 and the correction processing device 50 of the display system 100 are integrally formed devices.

[0178] The display device 600 includes a communication unit 61 , a control unit 62 , a storage unit 63 , an operation unit 64 , and a display unit 65 .

[0179] The communication unit 61 has a function or a structure equivalent to that of the communication unit 31 or the communication unit 51 .

[0180] The control unit 62 has the same function or structure as the display control unit 32 and the correction control unit 52. Specifically, the control unit 62 includes a threshold setting unit 62a, a determination unit 62b, a deviation amount calculation unit 62c, a correction amount calculation unit 62d, and a display control unit 62e. The threshold setting unit 62a, the determination unit 62b, the deviation amount calculation unit 62c, the correction amount calculation unit 62d, and the display control unit 62e in this embodiment correspond to the threshold setting unit 52a, the determination unit 52b, the deviation amount calculation unit 52c, the correction amount calculation unit 52d, and the display control unit 32 in the first to seventh embodiments, respectively.

[0181] The storage unit 63 corresponds to the storage unit 33 and the storage unit 53 , and stores the image data 330 and the threshold value data 530 .

[0182] The operation unit 64 is a user interface for inputting various operations by the user. For example, the operation unit 64 is a touch panel provided on the surface of the display unit 65. The operation unit 64 may be realized by a keyboard, buttons, switches, or a combination of these, in addition to the touch panel.

[0183] The display unit 65 includes, for example, a liquid crystal display or an organic EL display. The display unit 65 displays the image indicated by the image data 330 at, for example, a display position indicated by "reference position P0 + correction amount c" designated by the display control unit 62e.

[0184] The display device 600 may be connected to a projector or may be incorporated in the projector. The display unit 65 may have a function or structure equivalent to that of the projection device 10 .

[0185] As described above, the display device 600 includes an acquisition unit, a display unit 65, and a control unit 62. For example, the communication unit 61 is equivalent to an acquisition unit that acquires speed information indicating the speed of the moving body and change amount information indicating the amount of posture change of the moving body. The display unit 65 displays the image at a display position based on the reference position and the correction amount. The control unit 62 sets the correction amount based on the speed and the amount of posture change. Specifically, the control unit 62 sets a threshold value based on the speed. The control unit 62 sets the correction amount based on the posture change amount while the posture change amount is below the threshold, and returns the display position to the reference position while the posture change amount is greater than the threshold.

[0186] According to this embodiment, the same effects as those of the first embodiment can be obtained.

[0187] (Other embodiments)

[0188] As described above, the first to eighth embodiments are described as examples of the technology disclosed in this application. However, the technology disclosed in this application is not limited thereto, and can also be applied to embodiments in which changes, substitutions, additions, omissions, etc. are appropriately made. In addition, the various structural elements described in the first to eighth embodiments can also be combined to form new embodiments. Therefore, other embodiments are described below as examples.

[0189] In the first to seventh embodiments, the speed information is output from the information acquisition device 20 to the correction processing device 50 via the display processing device 30. However, the information acquisition device 20 may output the speed information to the correction processing device 50 directly.

[0190] In the above embodiment, the information acquisition device 20 acquires speed information indicating the speed of the vehicle 200 through the vehicle speed sensor 23. However, the speed of the vehicle 200 is not limited to being acquired through the vehicle speed sensor 23, and may also be acquired from other devices or by other methods. For example, the speed may also be calculated based on the position information obtained through the GPS module 21. The speed may also be calculated based on the acceleration of the vehicle 200 obtained from an acceleration sensor or the like.

[0191] In the above embodiment, an example is described in which the information acquisition device 20 includes a GPS module 21, a camera 22, and a vehicle speed sensor 23. However, the information acquisition device 20 may also include a distance sensor that measures the distance and direction from the vehicle 200 to the surrounding objects, and may output distance information indicating the measured distance and direction to the display processing device 30. The information acquisition device 20 may also include a navigation system. The information acquisition device 20 may also include one or more of a GPS module 21, a distance sensor, a camera 22, an image processing device, an acceleration sensor, a radar, an acoustic wave sensor, and a white line detection device of an ADAS (Advanced Driver-Assistance Systems). The GPS module 21, the distance sensor, the camera 22, and the vehicle speed sensor 23 that have the function of the information acquisition device 20 may be built into one device or may be independently installed in the vehicle 200. In addition, the so-called vehicle speed information includes all information that can determine the speed of the vehicle 200.

[0192] In the above embodiment, an example is described in which the posture detection device 40 includes the gyro sensor 41. However, the posture detection device 40 may also include an acceleration sensor that detects the acceleration of the vehicle 200, and outputs the detected acceleration as posture change information. The posture detection device 40 may also include a vehicle height sensor that detects the height from the road surface, and may output the detected height as posture change information. The posture detection device 40 may also include other well-known sensors. The posture detection device 40 may include one or more of the gyro sensor 41, the acceleration sensor, and the vehicle speed sensor. In this case, the gyro sensor 41, the acceleration sensor, and the vehicle height sensor that have the function of the posture detection device 40 may be built into one device, or may be independently installed on the vehicle 200.

[0193] In the first to seventh embodiments, the case where the projection device 10, the information acquisition device 20, the display processing device 30, the posture detection device 40, and the correction processing device 50 are respectively separate devices is exemplified. However, a plurality of devices may be formed integrally as one device. For example, the display processing device 30 and the correction processing device 50 may be formed integrally as one device. The information acquisition device 20 and the display processing device 30 may be formed integrally as one device. The posture detection device 40 and the correction processing device 50 may be formed integrally as one device. Separately formed devices can be connected to each other in a communicative manner by wire or wireless. In addition, the projection device 10, the information acquisition device 20, the display processing device 30, the posture detection device 40, and the correction processing device 50 may all be formed as one device. In this case, the communication units 31 and 51 may not exist.

[0194] In the above embodiment, the case where the moving object is a vehicle 200 such as a car is described. However, the moving object is not limited to the vehicle 200. The moving object may be a vehicle for people to ride, for example, an airplane or a ship. The moving object may also be a drone. The moving object may also be a vibrating object instead of a moving object.

[0195] In the first to seventh embodiments, the display system 100 is described as an example of a HUD system. However, the display system 100 may not be a HUD system. The display system 100 may also include a liquid crystal display or an organic EL display instead of the projection device 10. The display system 100 may also include a screen and a projector.

[0196] In the above embodiment, the display position is reset by comparing the angular velocity of the moving object with the threshold value, but the display position may be reset by comparing the angle of the moving object with the threshold value.

[0197] In the above embodiment, the case where the image is displayed in front of the moving body is described. However, the position where the image is displayed is not limited to the front. For example, the image may be displayed in the side direction or the rear of the moving body.

[0198] (Overview of Embodiments)

[0199] (1) The display system disclosed in the present invention comprises: an acquisition device for acquiring speed information indicating the speed of a moving body; a detection device for detecting the amount of change in the posture of the moving body; a display processing device for controlling the display position of an image based on a reference position and a correction amount; and a correction processing device for setting a correction amount based on the speed and the amount of change in posture, the correction processing device infers whether the correction amount is greater than a prescribed amount based on the speed and the amount of change in posture, and adjusts the correction amount based on the inference result.

[0200] Thus, it is possible to suppress the position deviation of the image with high accuracy. For example, regardless of the speed of the moving body, it is possible to reset when the vibration amplitude of the posture change exceeds the correction range (threshold), and to perform correction when the vibration amplitude of the posture change is within the correction range, thereby achieving high-accuracy position deviation correction that takes into account correction, reset, and quality during reset. For example, by changing the display position based on the posture change amount, it is possible to suppress the position deviation of the image based on the posture change with high accuracy. By returning the display position to the reference position, it is possible to suppress the position deviation of the image caused by the accumulation of correction errors.

[0201] (2) In the display system of (1), the correction processing device sets the first threshold value based on the speed, sets the correction amount based on the posture change amount while the posture change amount is less than the first threshold value, and the display processing device returns the display position to the reference position while the posture change amount is greater than the first threshold value. By returning the display position to the reference position, it is possible to suppress the position deviation of the image caused by the accumulation of the correction error.

[0202] (3) In the display system of (2), the correction processing device may set the first threshold value so that the first threshold value increases as the speed increases.

[0203] Thus, for example, even when the mobile body moves at a low speed and the vibration frequency is low, the display position can be returned to the reference position. In addition, even when the mobile body moves at a high speed and the vibration frequency is high, the display position can be changed by a correction amount based on the posture change amount.

[0204] (4) In the display system of (3), the posture change amount may be the angular velocity of the moving object.

[0205] This makes it possible to reset the display position in response to a sudden change in the posture of the moving object.

[0206] (5) In the display system of (4), the correction processing device may reset the correction amount to zero when the angular velocity is greater than the first threshold value.

[0207] (6) In the display system of (4), the correction processing device may reduce the correction amount by a predetermined amount so that the correction amount approaches zero when the angular velocity is greater than the first threshold value.

[0208] (7) In the display system of (4), the correction processing device may also reduce the correction amount by a specified amount so that the correction amount is close to zero when the angular velocity is greater than the first threshold value and the correction amount is greater than the second threshold value, and reset the correction amount to zero when the correction amount is less than the second threshold value.

[0209] (8) In the display system of (5), the correction processing device may also include: a first calculation unit, which calculates the angle of the moving body according to the angular velocity; and a second calculation unit, which converts the angle into a number of pixels to calculate the correction amount, wherein the second calculation unit calculates the correction amount based on the difference between the angle and a specified angle, or the difference between the number of pixels and a specified number of pixels, and sets the specified angle or the specified number of pixels so that the correction amount is zero when the angular velocity is greater than the first threshold value.

[0210] (9) In the display system of (5), the correction processing device may also include: a first calculation unit, which calculates the angle of the moving body based on the angular velocity; and a second calculation unit, which converts the angle into a number of pixels to calculate the correction amount, and the first calculation unit resets the angle to zero when the angular velocity is greater than a first threshold value.

[0211] (10) In the display system of (4), the display processing device may control the display position of the image based on the reference position, the correction amount, and the offset amount, and the correction processing device may set the offset amount based on the correction amount when the angular velocity is greater than the first threshold value.

[0212] (11) The display system of (1) may further include a projection device for projecting light representing an image.

[0213] (12) In the display system of (11), the moving object may be a vehicle, and the virtual image may be displayed in front of the windshield of the vehicle.

[0214] (13) The display device disclosed in the present invention comprises: an acquisition unit, which acquires speed information indicating the speed of a moving body and change information indicating the amount of change in the posture of the moving body; a display unit, which displays an image at a display position based on a reference position and a correction amount; and a control unit, which sets a correction amount based on the speed and the amount of change in posture. The control unit sets a threshold based on the speed, sets the correction amount based on the amount of change in posture when the amount of change in posture is below the threshold, and returns the display position to the reference position when the amount of change in posture is greater than the threshold.

[0215] (14) The display control method disclosed in the present invention is a display control method performed by a computing unit of a computer, the display control method comprising: a step of acquiring speed information indicating the speed of a moving body and change information indicating the change in posture of the moving body; and a step of controlling the display position of an image based on a reference position and a correction amount, wherein the step of controlling the display position sets a threshold based on the speed, sets a correction amount based on the posture change amount while the posture change amount is below the threshold, and returns the display position to the reference position while the posture change amount is greater than the threshold.

[0216] The display system, display device, and display control method described in all claims of the present disclosure can be implemented by cooperating with hardware resources, such as a processor, a memory, and a program.

[0217] Industrial Applicability

[0218] The present disclosure can be applied to a display device and a display system that display a virtual image in front of a windshield of a vehicle.

[0219] -Explanation of symbols-

[0220] 10 Projection device

[0221] 20 Information acquisition device

[0222] 21 GPS module

[0223] 22 Camera

[0224] 23 Vehicle speed sensor

[0225] 30 Display processing device

[0226] 31 Ministry of Communications

[0227] 32 Display control unit

[0228] 33 Storage

[0229] 40 Posture Detection Device

[0230] 41 Gyroscope sensor

[0231] 50 Correction Processing Device

[0232] 51 Ministry of Communications

[0233] 52 Correction Control Department

[0234] 52a Threshold setting unit

[0235] 52b Judgment Unit

[0236] 52c Deviation calculation unit

[0237] 52d Correction amount calculation unit

[0238] 53 Storage

[0239] 100 Display System

[0240] 600 display device.

Claims

1. A display system comprising: an acquisition device for acquiring speed information indicating the speed of the moving object; A detection device for detecting a change in posture of the moving body; A display processing device controls the display position of the image based on the reference position and the correction amount; and A correction processing device sets the correction amount based on the speed and the posture change amount, The correction processing device estimates whether the correction amount is equal to or greater than a predetermined amount based on the speed and the posture change amount, and adjusts the correction amount based on the estimation result.

2. The display system according to claim 1, wherein: The correction processing device sets a first threshold value based on the speed, and sets the correction amount based on the posture change amount while the posture change amount is equal to or less than the first threshold value. The display processing device returns the display position to the reference position while the posture change amount is greater than the first threshold value.

3. The display system according to claim 2, wherein: The correction processing device sets the first threshold value so that the first threshold value becomes larger as the speed becomes faster.

4. The display system according to claim 3, wherein: The posture change amount is the angular velocity of the moving object.

5. The display system according to claim 4, wherein: The correction processing device resets the correction amount to zero when the angular velocity is greater than the first threshold value.

6. The display system according to claim 4, wherein: The correction processing device reduces the correction amount by a predetermined amount so that the correction amount approaches zero when the angular velocity is greater than the first threshold value.

7. The display system according to claim 4, wherein: The correction processing device reduces the correction amount by a predetermined amount so that the correction amount approaches zero when the angular velocity is greater than the first threshold and the correction amount is greater than or equal to a second threshold, and resets the correction amount to zero when the correction amount is less than the second threshold.

8. The display system according to claim 5, wherein: The correction processing device comprises: a first calculation unit that calculates the angle of the moving object based on the angular velocity; and A second calculation unit converts the angle into a number of pixels to calculate the correction amount. The second calculation unit calculates the correction amount based on a difference between the angle and a predetermined angle or a difference between the number of pixels and a predetermined number of pixels, The second calculation unit sets the predetermined angle or the predetermined number of pixels so that the correction amount becomes zero when the angular velocity is greater than the first threshold value.

9. The display system according to claim 5, wherein: The correction processing device comprises: a first calculation unit that calculates the angle of the moving object based on the angular velocity; and A second calculation unit converts the angle into a number of pixels to calculate the correction amount. The first calculation unit resets the angle to zero when the angular velocity is greater than the first threshold value.

10. The display system according to claim 4, wherein: The display processing device controls the display position of the image based on the reference position, the correction amount, and the offset amount. The correction processing device sets the offset amount based on the correction amount when the angular velocity is greater than the first threshold value.

11. The display system according to claim 1, wherein: The display system further includes a projection device for projecting light representing the image.

12. The display system according to claim 11, wherein: The mobile object is a vehicle, The image is a virtual image displayed in front of the windshield of the vehicle.

13. A display device comprising: an acquisition unit that acquires speed information indicating a speed of a moving object and change amount information indicating a change amount of a posture of the moving object; A display unit that displays an image at a display position based on a reference position and a correction amount; and The control unit sets the correction amount based on the speed and the posture change amount. The control unit sets a threshold value based on the speed, The control unit sets the correction amount based on the posture change amount while the posture change amount is equal to or less than the threshold value, The control unit returns the display position to the reference position while the posture change amount is greater than the threshold value.

14. A display control method, which is a display control method performed by a computing unit of a computer, the display control method comprising: a step of acquiring speed information indicating the speed of a moving object and change amount information indicating the change amount of the posture of the moving object; and The step of controlling the display position of the image based on the reference position and the correction amount, The step of controlling the display position comprises: A threshold is set based on the speed, setting the correction amount based on the posture change amount while the posture change amount is equal to or less than the threshold value, While the posture change amount is greater than the threshold value, the display position is returned to the reference position.

Images