Display System

Abstract

A display system of the present disclosure comprises: a display processing device that controls the display of displayed content; an image correction unit that performs image correction processing which involves changing the display position of the displayed content on the basis of pre-stored image correction data; a posture detection device that detects an amount of variation in the posture of a moving body; a vibration correction processing device that calculates a vibration correction amount for the display position of the displayed content on the basis of the amount of variation in the posture of the moving body; and a processing switching unit that switches the sequential order of processing between image correction processing that involves changing the display position of the displayed content and vibration correction processing that corrects the display position of the displayed content on the basis of the vibration correction amount. The display processing device controls the display position of the displayed content by implementing the image correction processing and the vibration correction processing on the basis of the order of the processing switched by the processing switching unit.

Description

[Technical field]

[0001] The present disclosure relates to a display system that controls a display position of a display content in accordance with the movement of a mobile object. [Background technology]

[0002] Patent Document 1 discloses a vehicle information projection system that uses a head-up display (HUD) device to perform an augmented reality (AR) display. The HUD device projects light representing a virtual image onto the windshield of the vehicle, allowing the viewer, who is the vehicle occupant, to view the virtual image together with the real scene outside the vehicle. For example, a virtual image representing the vehicle's guide route is displayed in association with a display object (e.g., a road) in the real scene. This allows the occupant to check the guide route while viewing the real scene. The vehicle information projection system described in Patent Document 1 includes a vehicle speed sensor and corrects the display position of the virtual image according to the acceleration. This suppresses the occurrence of positional deviation of the virtual image when the vehicle suddenly decelerates or accelerates. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-101311 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a display system that suppresses misalignment of the display position of display content. [Means for solving the problem]

[0005] The display system of the present disclosure includes a display processing device that controls the display of display content, an image correction unit that performs image correction processing involving a change in the display position of the display content based on pre-stored image correction data, a posture detection device that detects an amount of posture change of a moving body, a vibration correction processing device that calculates a vibration correction amount for the display position of the display content based on the amount of posture change of the moving body, and a process switching unit that switches the order of the image correction processing involving a change in the display position of the display content and the vibration correction processing that corrects the display position of the display content based on the vibration correction amount, and the display processing device controls the display position of the display content by performing the image correction processing and the vibration correction processing based on the process order switched by the process switching unit.

[0006] These general and specific aspects may be realized by a system, a method, and a computer program, as well as combinations thereof. Effect of the Invention

[0007] According to the display system of the present disclosure, it is possible to suppress misalignment of the display position of the display content. [Brief description of the drawings]

[0008] [Figure 1] Diagram to explain head-up display (HUD) [Diagram 2] FIG. 1 is a block diagram showing an internal configuration of a display system according to a first embodiment. [Figure 3A] A diagram showing an example when the vehicle is not tilted [Figure 3B] A diagram showing an example of the actual view seen through the windshield [Figure 3C] FIG. 13 is a diagram showing an example in which display content is displayed at a reference position. [Figure 3D] A diagram showing an example of an augmented reality (AR) display [Figure 4A] A diagram showing the vehicle's backward tilted position. [Figure 4B] FIG. 13 is a diagram for explaining an example in which positional deviation of display content occurs when a vehicle is in a backward tilting position; [Figure 4C] FIG. 13 is a diagram showing a display example of a display content after vibration correction processing; [Diagram 5] An image diagram for explaining image correction processing [Figure 6A] FIG. 13 is a diagram showing an example of multiple partitioned regions in a display region. [Figure 6B] FIG. 13 is a diagram showing an example of image correction data. [Figure 7] 1 is a flowchart showing a display process in the first embodiment. [Figure 8] Flowchart showing vibration correction processing [Figure 9] Flowchart showing image correction processing [Figure 10] 1 is a flowchart showing a calculation process of a vibration correction amount in the first embodiment. [Figure 11] FIG. 13 is a block diagram showing the internal configuration of a display system according to a second embodiment. [Figure 12] 11 is a flowchart showing a display process in the second embodiment. [Figure 13] 11 is a flowchart showing a calculation process of a vibration correction amount in the second embodiment. [Figure 14A] 14A is an explanatory diagram showing the effect of an offset removal filter, in which FIG. 14A(a) shows an offset error in the amount of fluctuation, and FIG. 14A(b) shows the amount of fluctuation after the offset error has been removed. [Figure 14B] FIG. 14B is an explanatory diagram showing an example of counting, in which FIG. 14B(a) shows an example of the vibration of the fluctuation amount, and FIG. 14B(b) is an explanatory diagram showing a counted flag. [Figure 15] FIG. 13 is a block diagram showing the internal configuration of a display system according to a third embodiment. [Figure 16A] FIG. 13 is a diagram showing an example of an initial position of a display content. [Figure 16B] FIG. 13 is a diagram showing another example of the initial position of the display content. [Figure 17] FIG. 13 is a diagram showing an example of an image correction error occurring when a vibration correction process is performed after an image correction process; [Figure 18] 13 is a flowchart showing a calculation process of a vibration correction amount in the third embodiment. [Figure 19A]FIG. 13 is a diagram showing an example of the display position of a display content in a display area before and after vibration correction; [Figure 19B] FIG. 13 is a diagram for explaining a first image correction value before vibration correction and a second image correction value after vibration correction. [Figure 20] FIG. 13 is a diagram showing an example of a first correction value table including first correction values ​​for image correction assigned to amounts of deviation from an initial position of a display content; [Figure 21] 13 is a flowchart showing a calculation process of a vibration correction amount according to a modified example of the third embodiment. [Figure 22] FIG. 13 is a diagram showing an example of a second correction value table including second correction values ​​of image correction assigned to deviation correction amounts from an initial position of a display content; [Figure 23] FIG. 13 is a block diagram showing the internal configuration of a display system according to a fourth embodiment. [Figure 24] 13 is a flowchart showing a calculation process of a vibration correction amount in the fourth embodiment. [Diagram 25] 13 is a flowchart showing a calculation process of a vibration correction amount according to a modification of the fourth embodiment. [Figure 26] 13 is a flowchart showing a display process in the fifth embodiment. [Figure 27] 13 is a flowchart showing a calculation process of a vibration correction amount in the fifth embodiment. [Figure 28] 13 is a flowchart showing a calculation process of a vibration correction amount according to a modified example of the fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] (Findings on which this disclosure is based) It is known that a display system electronically corrects the display position of display content (e.g., arrows, figures, characters, etc.) displayed in a display area according to the state (e.g., posture) of a moving object detected based on the output of a sensor. For example, the display system performs a vibration correction process that corrects the display position of the display content based on the vibration of the moving object detected by a sensor.

[0010] In addition, in a display system, when display content is displayed in a display area, distortion or rotational deviation may occur in the displayed image. For example, when display content is displayed in front of the windshield of a moving vehicle, the display content may be distorted when reflected by the windshield. Alternatively, the display content may be displayed tilted due to variations during manufacturing of the display system. For this reason, the display system performs image correction processing to change the display position of the display content in order to correct distortion or rotational deviation of the display content.

[0011] For example, the display system acquires information such as distortion and rotational deviation when the display content is displayed in the display area during manufacturing or design, and stores image correction data calculated based on the information in a storage unit. The image correction data includes information on a correction value for correcting deviations in the display position of the display content, such as distortion and rotational deviation. The display system corrects distortion, rotational deviation, etc. by performing image correction processing based on the image correction data previously stored in the storage unit.

[0012] In such a display system, the vibration correction process and the image correction process are performed in a fixed order.

[0013] However, in a display system in which the order of processing is fixed, there is a problem that the accuracy of correcting the display position of the displayed content decreases due to factors such as the traveling conditions of the moving object and the processing time of the image correction process. This is a problem that the present inventors have newly discovered.

[0014] As an example, a display system that performs an image correction process after performing a vibration correction process will be described. In such a display system, when the processing time of the image correction process becomes longer, the delay time from the completion of the change in the display position of the display content by the vibration correction process to the display becomes longer. In addition, in order to achieve a tolerable correction error by the correction process, there is a limit to the allowable delay time. For example, even if the delay time from the completion of the change in the display position of the display content by the vibration correction process to the display does not change, as the frequency of the generated vibration increases, the allowable delay time for suppressing the vibration correction error in the vibration correction process to a certain level or less becomes shorter. As a result, the vibration correction error increases, and the correction accuracy of the vibration correction process decreases. In order to suppress the decrease in the correction accuracy, it is effective to reduce the delay time.

[0015] One method for reducing delay time when the image correction process takes a long time or when high-frequency vibration occurs is to perform a vibration correction process after performing an image correction process. However, the accuracy of image correction is higher when the image correction process is performed after the vibration correction process. If the vibration correction process is always performed after the image correction process, it becomes difficult to improve the accuracy of image correction. Furthermore, when the image correction process takes a short time or when high-frequency vibration does not occur, performing the vibration correction process after performing the image correction process has little effect on improving the accuracy of vibration correction. Therefore, when the image correction process takes a short time or when high-frequency vibration does not occur, performing the image correction process after performing the vibration correction process can improve the accuracy of image correction.

[0016] Therefore, the present inventors have found that the order of the vibration correction process and the image correction process can be switched to solve the above-mentioned problem. For example, when the processing time of the image correction process is long, the order of the processes can be switched so that the vibration correction process is performed after the image correction process, thereby shortening the delay time from the completion of the change in the display position of the display content by the vibration correction process to the display. This can reduce the vibration correction error and suppress the deterioration of the correction accuracy of the display position of the display content. On the other hand, when the processing time of the image correction process is short, performing the image correction process after the vibration correction process can improve the correction accuracy of the image correction process while reducing the deterioration of the correction accuracy due to the vibration correction process.

[0017] The display system of the present disclosure switches the order of the vibration correction process and the image correction process depending on the traveling conditions of the moving object, the processing time of the image correction process, etc. This makes it possible to suppress a decrease in the correction accuracy of the displayed content. As a result, it is possible to suppress a position shift of the display position of the displayed content.

[0018] (First embodiment) Hereinafter, a first embodiment will be described with reference to the drawings. In the first embodiment, a case will be described in which a moving body is a vehicle such as an automobile, and a display system is a head-up display (HUD) system that displays display content in front of a windshield of the vehicle.

[0019] 1. Display System Configuration The configuration of a display system according to this embodiment will be described with reference to FIGS.

[0020] Fig. 1 is a diagram for explaining the HUD system. In Fig. 1, the roll axis of the vehicle 200 is the X axis, the pitch axis of the vehicle 200 is the Y axis, and the yaw axis of the vehicle 200 is the Z axis. That is, the X axis is perpendicular to the Y axis and the Z axis and is an axis along the line of sight of the occupant D viewing the display content Iv. The Y axis is an axis along the left-right direction as seen by the occupant D viewing the display content Iv. The Z axis is an axis along the height direction of the vehicle 200.

[0021] The display system 100 of this embodiment is a HUD system that performs so-called augmented reality (AR) display by superimposing a display content Iv on an actual scene in front of a windshield 210 of a vehicle 200. The display content Iv is a content indicating predetermined information displayed in a display area 220. For example, the display content Iv is a figure and a character indicating a route for guiding to a destination, an estimated arrival time to the destination, a traveling direction, a speed, various warnings, and the like. In this embodiment, an arrow indicating a route for guiding to a destination is described as an example of the display content Iv. The display system 100 is installed in a vehicle 200, and projects a display light Lc representing the display content Iv into a display area 220 of a windshield 210 of the vehicle 200. In this embodiment, the display area 220 is a partial area of ​​the windshield 210. Note that the display area 220 may be the entire area of ​​the windshield 210. The display light Lc is reflected by the windshield 210 toward the inside of the vehicle. As a result, the occupant D inside the vehicle 200 visually recognizes the reflected display light Lc as display content Iv in front of the vehicle 200.

[0022] The display system 100 includes a projection device 10, an information acquisition device 20, a display processing device 30, an attitude detection device 40, and a vibration correction processing device 50.

[0023] The projection device 10 projects display light Lc representing the display content Iv into the display area 220. The projection device 10 includes, for example, a liquid crystal display element that displays an image of the display content Iv, a light source such as an LED that illuminates the liquid crystal display element, and a mirror and a lens that reflect the display light Lc of the image displayed by the liquid crystal display element into the display area 220. The projection device 10 is installed, for example, in the dashboard of the vehicle 200.

[0024] The information acquisition device 20 acquires information indicating the position of the vehicle 200. Specifically, the information acquisition device 20 measures the position of the vehicle 200 and generates position information indicating the position. The information acquisition device 20 outputs vehicle-related information including the position information of the vehicle 200.

[0025] The display processing device 30 controls the display of the display content Iv based on the vehicle-related information obtained from the information acquisition device 20, and outputs image data of the display content Iv to the projection device 10. The display processing device 30 may control the display of the display content Iv based on the display timing (display time) of the display content Iv, or a combination of the vehicle-related information and the display timing. The display timing is, for example, repeating a display for 10 seconds and a non-display for 1 second. In addition, the display processing device 30 performs an image correction process involving a change in the display position of the display content Iv based on the image correction data, and corrects display deviations such as distortion and rotational deviation of the display content Iv.

[0026] The attitude detection device 40 detects an attitude change of the vehicle 200. Specifically, the attitude detection device 40 detects an amount of change in attitude of the vehicle 200. The amount of change in attitude is an angular velocity.

[0027] In this embodiment, the attitude detection device 40 includes, for example, a gyro sensor 41 (see FIG. 2) that detects an angular velocity. The gyro sensor 41 outputs the detected angular velocity to the vibration correction processing device 50 as an attitude change amount indicating the attitude change of the vehicle 200.

[0028] The vibration correction processing device 50 calculates a vibration correction amount for the display position of the display content Iv based on the attitude change of the vehicle 200 detected by the attitude detection device 40. The vibration correction processing device 50 outputs the calculated vibration correction amount to the display processing device 30. As a result, the display position of the display content Iv is corrected within the display area 220 in accordance with changes in the actual scene seen through the windshield 210.

[0029] FIG. 2 is a block diagram showing the internal configuration of the display system 100. As shown in FIG.

[0030] In this embodiment, the information acquisition device 20 includes a GPS (Global Positioning System) module 21 that detects a position indicating the current location of the vehicle 200 in a geographic coordinate system. Specifically, the GPS module 21 receives radio waves from GPS satellites and measures the latitude and longitude of the received point. The GPS module 21 generates position information indicating the measured latitude and longitude. The information acquisition device 20 outputs vehicle-related information including the position information to the display processing device 30.

[0031] The display processing device 30 includes a communication unit 31, a display control unit 32, a storage unit 33, and an image correction unit .

[0032] The communication unit 31 includes a circuit for communicating with an external device in accordance with a predetermined communication standard. The predetermined communication standard includes, for example, LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), USB, HDMI (registered trademark), CAN (controller area network), and SPI (Serial Peripheral Interface).

[0033] The display control unit 32 can be realized by a semiconductor element or the like. The display control unit 32 can be configured by, for example, a microcomputer, a CPU, an MPU, a GPU, a DSP, an FPGA, or an ASIC. The functions of the display control unit 32 may be configured by hardware alone, or may be realized by combining hardware and software. The display control unit 32 realizes a predetermined function by reading data and programs stored in the storage unit 33 and performing various arithmetic processing.

[0034] The display control unit 32 includes a process switching unit 32a.

[0035] The process switching unit 32a switches the order of the image correction process and the vibration correction process. The image correction process is image correction accompanied by a change in the display position of the display content Iv performed by the image correction unit 34. The vibration correction process is vibration correction that corrects the display position of the display content Iv based on the vibration correction amount calculated by the vibration correction processing device 50. The process switching unit 32a switches the order of the image correction process and the vibration correction process based on a predetermined condition.

[0036] In this embodiment, the process switching unit 32a acquires an image processing time Td of the image correction process, and switches the order of the image correction process and the vibration correction process based on the image processing time Td. The image processing time Td means a processing time required to change the display position of the display content Iv by the image correction process. For example, the image processing time Td may be a predetermined value, or may be a value that changes for each image correction process target. Alternatively, the image processing time Td may be calculated based on the information amount of the display content Iv on which the image correction process is performed and / or the image correction value U. For example, when the image processing time Td is equal to or less than the first threshold T1, the process switching unit 32a switches the order of the processes so that the image correction process is performed after the vibration correction process. Also, when the image processing time Td is greater than the first threshold T1, the process switching unit 32a switches the order of the processes so that the image correction process is performed after the vibration correction process.

[0037] The storage unit 33 is a storage medium that stores programs and data necessary to realize the functions of the display processing device 30. The storage unit 33 can be realized by, for example, a hard disk drive (HDD), an SSD, a RAM, a DRAM, a ferroelectric memory, a flash memory, a magnetic disk, or a combination of these.

[0038] A plurality of image data 33i representing the display content Iv are stored in the storage unit 33. The storage unit 33 also stores image correction data 33g used in the image correction process.

[0039] The image correction data 33g includes, for example, an image correction value for correcting a display deviation such as a distortion and / or a rotation deviation of an image representing the display content Iv. The image correction value is, for example, known data calculated based on the distortion and / or the rotation deviation of an image measured by an inspection during the manufacture of the display system 100, and is a static value.

[0040] In this specification, the distortion refers to, for example, a distortion that occurs when the display content Iv is displayed in front of the windshield 210 of the vehicle 200 and the display content Iv is reflected by the windshield 210. The rotational deviation refers to a deviation that occurs due to variations in installation, such as when the projection device 10 is installed at an angle with respect to the windshield 210 onto which light is projected from the projection device 10.

[0041] For example, the image correction data 33g may store image correction values ​​assigned to each of the divided regions by dividing the display region 220 into a plurality of divided regions. Specifically, the image correction data 33g may store an image correction table indicating the image correction values ​​assigned to each of the divided regions.

[0042] In the present embodiment, the image correction data 33g includes an image correction value for correcting distortion and / or rotational deviation, but is not limited thereto. The image correction data 33g may include an image correction value for correcting a display deviation that causes a change in the display position of the display content Iv, other than the distortion and / or rotational deviation.

[0043] The image correction unit 34 performs image correction processing accompanied by a change in the display position of the display content Iv, based on image correction data 33g pre-stored in the storage unit 33. The image correction processing accompanied by a change in the display position of the display content Iv includes, for example, correction of image distortion and / or correction of image rotational shift, etc. The image correction unit 34 corrects distortion and / or rotational shift, etc., of the display content Iv projected by the projection device 10 onto the windshield 210 of the vehicle 200.

[0044] For example, the image correction unit 34 reads out image correction data 33g stored in advance in the storage unit 33, and determines an image correction value for correcting a display shift of the display position of the display content Iv caused by distortion and / or rotation shift, etc. The image correction unit 34 determines the image correction value using an image correction table of the image correction data 33g.

[0045] The image correction value determined by the image correction unit 34 is output to the display control unit 32.

[0046] The image correction unit 34 may be included in the display control unit 32.

[0047] The display control unit 32 determines the display content Iv to be displayed in the display area 220 based on the vehicle-related information obtained from the information acquisition device 20. The display control unit 32 reads out image data 33i of the determined display content Iv from the storage unit 33 and outputs it to the projection device 10.

[0048] The display control unit 32 acquires information indicating a display reference position from an external device (not shown) via the communication unit 31. The display reference position is a position within the display area 220 that serves as a reference for displaying the display content Iv.

[0049] The display control unit 32 acquires content information indicating information on the display content Iv. The content information includes, for example, information on the shape, size, display edge, and display reference position of the display content Iv. In this specification, information on the display content including the display reference position and content information (for example, information on the size of the display content) is referred to as content display information. The content display information includes at least a characteristic portion that determines the shape of the display content. The characteristic portion is, for example, a pixel position of the content display. The content display information is stored in the storage unit 33 together with the image data 33i. In addition, the display control unit 32 acquires vehicle-related information from the information acquisition device 20 via the communication unit 31, and determines image data to be used from the multiple image data 33i stored in the storage unit 33 based on the vehicle-related information. As a result, the display control unit 32 acquires information indicating the shape, size, display edge, and the like of the display content Iv.

[0050] The display control unit 32 outputs the content display information to the vibration correction processing device 50 .

[0051] The display control unit 32 acquires an image correction value of the display content Iv from the image correction unit 34, and changes the display position of the display content Iv based on the image correction value. The display control unit 32 also acquires a vibration correction amount from the vibration correction processing device 50. The display control unit 32 sets the display position of the display content Iv based on the display reference position and the vibration correction amount.

[0052] The attitude detection device 40 includes a gyro sensor 41 that detects an angular velocity. The gyro sensor 41 outputs the detected angular velocity to the vibration correction processing device 50 as an attitude change amount that indicates the attitude change of the vehicle 200.

[0053] The output of the attitude detection device 40 may be processed by the attitude detection device 40, the deviation amount calculation section 52a of the vibration correction processing device 50, or another configuration.

[0054] The vibration correction processing device 50 includes a communication unit 51 , a correction control unit 52 , and a storage unit 53 .

[0055] The communication unit 51 includes a circuit for communicating with an external device in accordance with a predetermined communication standard. The predetermined communication standard includes, for example, LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), USB, HDMI (registered trademark), CAN (controller area network), and SPI (Serial Peripheral Interface).

[0056] The correction control unit 52 can be realized by a semiconductor element or the like. The correction control unit 52 can be configured by, for example, a microcomputer, a CPU, an MPU, a GPU, a DSP, an FPGA, or an ASIC. The functions of the display control unit 32 may be configured by hardware alone, or may be realized by combining hardware and software. The correction control unit 52 realizes a predetermined function by reading data and programs stored in a storage unit (not shown) in the vibration correction processing device 50 and performing various arithmetic processing.

[0057] The correction control unit 52 includes, as functional components, a deviation amount calculation unit 52a and a correction amount calculation unit 52b.

[0058] The deviation amount calculation unit 52a calculates the attitude (angle deviation amount) of the vehicle 200 based on the attitude change amount output by the attitude detection device 40. The angle deviation amount is an attitude angle with respect to a reference attitude state of the moving body. The reference attitude state of the moving body is, for example, a vehicle state in a stationary state placed in a horizontal state. For example, the deviation amount calculation unit 52a calculates an angle (pitch angle) around the pitch axis of the vehicle 200 by integrating the angular velocity detected by the gyro sensor 41. This makes it possible to calculate the deviation amount (angle) of the vehicle 200 in the rotation direction centered on the Y axis (pitch axis) shown in FIG. 1. Note that, although the pitch angle is calculated in this embodiment, the yaw angle or roll angle may be calculated. For example, the angles around the X axis, the Y axis, and the Z axis may all be calculated. In a reference attitude state of the moving body, the pitch angle, the yaw angle, and the roll angle are each 0°. In this manner, the deviation amount, which is an angle with respect to the three axial directions, may be calculated. When the attitude detection device 40 outputs the pitch angle, yaw angle, and roll angle, these values ​​may be processed as deviation amounts, or may be processed as deviation amounts by performing calculation processing other than integration.

[0059] The correction amount calculation unit 52b calculates a correction amount (vibration correction amount) for the display position of the display content Iv based on the attitude (angle deviation amount) of the vehicle 200. The correction amount is indicated by, for example, the number of pixels in the Y-axis direction and the Z-axis direction. Specifically, the correction amount calculation unit 52b converts, for example, the pitch angle and yaw angle deviation amounts calculated by the deviation amount calculation unit 52a from angles to the number of pixels, and determines a correction amount that restores the number of pixels by the deviation. For example, for the roll angle, the correction amount calculation unit 52b determines a correction amount that restores the roll angle deviation amount while leaving the angle unchanged. The correction amount calculation unit 52b outputs the calculated correction amount to the display processing device 30.

[0060] As described above, the display processing device 30 and the vibration correction processing device 50 communicate bidirectionally via the communication units 31 and 51. The display processing device 30 outputs content display information to the vibration correction processing device 50. The vibration correction processing device 50 outputs correction information indicating the amount of vibration correction to the display processing device 30.

[0061] 2. AR display AR display will be described with reference to FIG. 3A to FIG. 3D. FIG. 3A shows an example when the vehicle 200 is not tilted. FIG. 3B shows an example of an actual scene seen from the windshield 210 of the vehicle 200 shown in FIG. 3A. FIG. 3C shows an example of the display content Iv seen from the display area 220. FIG. 3D shows an example in which the display content Iv shown in FIG. 3C is superimposed on the actual scene shown in FIG. 3B. The display system 100 superimposes the display content Iv shown in FIG. 3C on the actual scene shown in FIG. 3B. The display reference position P0 of the display content Iv is a position determined based on the type of the display content Iv, the state (position and posture) of the vehicle 200, map data, and the like. For example, when the display object 230 is a driving lane and the display content Iv is an arrow indicating the traveling direction, the display reference position P0 is the display position on the liquid crystal when the tip of the arrow points to the center of the driving lane. The display reference position P0 is set at the position of a pixel on the liquid crystal display corresponding to the values ​​of the Y coordinate and Z coordinate in the display area 220 in FIG. 3C, for example. The display reference position P0 is acquired from an external device. The external device can be configured with, for example, a microcomputer, a CPU, an MPU, a GPU, a DSP, an FPGA, or an ASIC, and a GPS module 21. The function of the external device may be configured with only hardware, or may be realized by combining hardware and software. The display reference position P0 output from the external device may change based on the number of occupants, changes in load, and changes in posture due to a decrease in gasoline, etc., and therefore may differ from the display reference position acquired initially, for example. Therefore, the display processing device 30 may change the display reference position P0 acquired from the external device based on changes in posture due to the number of occupants, changes in load, and a decrease in gasoline, etc. The display processing device 30 may set the display reference position P0 based on vehicle-related information and map data, etc. The display processing device 30 may set the size of the display content Iv based on the vehicle-related information.

[0062] 3. Vibration correction processing for display content The vibration correction process for displaying the display content Iv will be described with reference to Figures 4A to 4C. Figure 4A shows an example of a state in which the vehicle 200 is in a forward-leaning posture. Figure 4B shows an example in which the display position of the display content Iv is shifted from the display target 230 in response to a change in posture of the vehicle 200. Figure 4C shows the display position of the display content Iv after vibration correction.

[0063] The vehicle 200 may tilt due to unevenness of the road surface, sudden acceleration or sudden deceleration of the vehicle 200, and the like. For example, when the vehicle 200 suddenly decelerates, the vehicle 200 assumes a forward-tilted posture as shown in FIG. 4A. In this case, as shown in FIG. 4B, the position of the display target 230 seen from the windshield 210 varies according to the inclination of the vehicle 200. Therefore, when the display content Iv is displayed at the display reference position P0, the display content Iv is displaced from the display target 230. For example, as shown in FIG. 4B, the tip of the arrow is within the oncoming lane 231. Therefore, the display system 100 adjusts the display position of the display content Iv in a direction to return the displacement according to the posture of the vehicle 200. Specifically, as shown in FIG. 4C, the vibration correction processing device 50 calculates the vibration correction amount C so that the display position is at a position P1 where there is no displacement of the display position due to the angle of the vehicle 200. That is, the display processing device 30 sets the display position of the display content Iv to "display reference position P0+vibration correction amount C". This allows the projection device 10 to display the display content Iv at the position P1 corresponding to the display target 230. In this way, even if the vehicle 200 is tilted, the display position of the display content Iv is changed from the display reference position P0 based on the vibration correction amount C, so that the display content Iv can be displayed at the position P1 corresponding to the display target 230 in the actual scene.

[0064] 4. Image correction processing for displaying content With reference to FIG. 5, the image correction process for displaying the display content Iv will be described. FIG. 5 shows an image diagram for explaining the image correction process. In the example shown in FIG. 5, the distortion correction process will be described when a distortion occurs in the vertical direction of the display area 220. As shown in FIG. 5, when the display area 220 is divided into a plurality of partitioned areas, the sizes of the plurality of partitioned areas differ due to the distortion. In the example shown in FIG. 5, in the vertical direction of the display area 220, the sizes of the upper areas L11, L12 and the lower areas L15, L16 are smaller than the partitioned areas when there is no distortion. Also, the sizes of the central areas L13, L14 are larger than the partitioned areas when there is no distortion. In this case, the sizes of the display contents Iv11-Iv16 displayed in the respective partitioned areas change, and the display positions P11-P16 of the tips of the arrows, which are characteristic parts of the display contents Iv11-Iv16, are shifted from the superimposed positions Q11-Q16.

[0065] The image correction unit 34 performs image correction processing involving a change in the display position of the display content Iv based on the image correction data 33g stored in advance in the storage unit 33. Specifically, the image correction unit 34 determines an image correction value using an image correction table of the image correction data 33g, and outputs the image correction value to the display control unit 32. In the example shown in FIG. 5, the image correction unit 34 determines the image correction value so that the display contents Iv11, Iv12, Iv15, and Iv16 are increased in the vertical direction in the upper regions L11 and L12 and the lower regions L15 and L16. Also, the image correction unit 34 determines the image correction value so that the display contents Iv13 and Iv14 are decreased in the vertical direction in the central regions L13 and L14. As a result, the display control unit 32 controls the display positions P11 to P16 of the display contents Iv11 to Iv16 based on the determined image correction value, and can superimpose them on the superimposition positions Q11 to Q16.

[0066] The determination of the image correction value will be described with reference to Fig. 6A and Fig. 6B. Fig. 6A shows an example of a plurality of divided regions R11 to R64 in the display region 220. As shown in Fig. 6A, the display region 220 in which the display content Iv is displayed has a plurality of divided regions R11 to R64. For example, the plurality of divided regions R11 to R64 are divided vertically and horizontally in a mesh pattern. In the example shown in Fig. 6A, the display region 220 is divided into six divided regions R11 to R64 vertically and four divided regions horizontally. Note that the number and arrangement of the divided regions in the display region 220 are not limited to this. It is sufficient that the display region 220 has a plurality of divided regions.

[0067] Fig. 6B shows an example of the image correction data 33g. As shown in Fig. 6B, the image correction data 33g has an image correction table including image correction values ​​U11-U64 assigned to each of the multiple divided regions R11-R64 of the display region 220. In the image correction table, the image correction values ​​U11-U64 of the image correction data 33g correspond to the multiple divided regions R11-R64 of the display region 220, respectively. For example, when the display content Iv is displayed in the divided region R22, the image correction value U22 is adopted as the image correction value.

[0068] 5. Operation of the display processing device The operation of the display control unit 32 of the display processing device 30 will be described with reference to Fig. 7. Fig. 7 shows the display process performed by the display control unit 32 of the display processing device 30. The display process shown in Fig. 7 is started, for example, when the engine of the vehicle 200 is started, or when a button for instructing the start of display of the display content Iv is operated.

[0069] The display control unit 32 acquires vehicle-related information including position information of the vehicle 200 from the information acquisition device 20 (S101). The display control unit 32 determines whether or not to display the display content Iv corresponding to the display target 230 based on the vehicle-related information (S102).

[0070] When it is determined that the display content Iv is to be displayed (Yes in S103), the display control unit 32 acquires information indicating the display reference position P0 of the display content Iv from the external device and acquires content information from the storage unit 33 (S104). The display control unit 32 outputs the content display information to the vibration correction processing device 50 (S105). For example, the content display information includes the display reference position and the content information.

[0071] The display control unit 32 acquires the image processing time Td (S106). For example, the process switching unit 32a of the display control unit 32 calculates the image processing time Td based on the display reference position P0, the content information, and the image correction value U. Note that an approximate value of the image processing time Td may be stored in advance in the content information. In this case, the display control unit 32 may use the approximate value stored in the content information as the image processing time Td as is, or may calculate the image processing time Td based on the approximate value.

[0072] The process switching unit 32a switches the order of the image correction process and the vibration correction process based on the image processing time Td. In the present embodiment, the process switching unit 32a switches the order of the image correction process and the vibration correction process based on the image processing time Td and the first threshold T1.

[0073] The process switching unit 32a determines whether the image processing time Td is equal to or less than the first threshold T1 (S107). If the image processing time Td is equal to or less than the first threshold T1 (Yes in S107), the process switching unit 32a switches the order of processes so that the image correction process is performed after the vibration correction process. That is, if the image processing time Td is equal to or less than the first threshold T1, the display processing device 30 first performs the vibration correction process (S108). The display processing device 30 performs the vibration correction process and then the image correction process (S109).

[0074] If the image processing time Td is not equal to or less than the first threshold T1 (No in S107), the process switching unit 32a switches the order of processes so that the image correction process is performed after the vibration correction process. That is, if the image processing time Td is greater than the first threshold T1, the display processing device 30 performs the image correction process (S110). The display processing device 30 performs the image correction process and then the vibration correction process (S111).

[0075] 8 shows the vibration correction process performed by the display control unit 32 of the display processing device 30. The vibration correction process shown in FIG. 8 shows details of steps S108 and S111 shown in FIG.

[0076] The display processing device 30 acquires the vibration correction amount C from the vibration correction processing device 50 (S120). The display control unit 32 changes the setting of the display position of the display content Iv based on the display reference position P0, the content information, and the vibration correction amount C (S121).

[0077] For example, when only vibration correction processing is performed, i.e., when the image correction value U is "0", the display control unit 32 reads out image data 33i of the display content Iv corresponding to the display target from the memory unit 33, and sets the display position of the display content Iv to "display reference position P0 + vibration correction amount C".

[0078] When performing vibration correction processing after image correction processing, the display control unit 32 reads out image data 33i of the display content Iv from the memory unit 33, and sets the image that has been subjected to image correction processing based on the image correction value U to the "display reference position P0 + vibration correction amount C."

[0079] When only image correction processing is performed, i.e., when the vibration correction amount C is "0", the display control unit 32 reads out image data 33i of the display content Iv from the memory unit 33, and sets the image that has been subjected to image correction processing based on the image correction value U to the "display reference position P0".

[0080] When performing image correction processing after performing vibration correction processing, the display control unit 32 reads out image data 33i of the display content Iv from the storage unit 33, and sets the read image to "display reference position P0+vibration correction amount C." In the image correction processing described later, the image correction processing is performed based on the image correction value U at the position of the image changed in the vibration correction processing.

[0081] 9 shows image correction processing performed by the display control unit 32 of the display processing device 30. The image correction processing shown in FIG. 9 shows details of steps S109 and S110 shown in FIG.

[0082] The display control unit 32 acquires information indicating the image correction value U of the display content Iv from the image correction unit 34 (S130). The display control unit 32 changes the setting of the display position of the display content Iv based on the display reference position P0, the content information, and the image correction value U (S131). For example, the display control unit 32 performs correction such as enlarging, reducing, and / or rotating the image using the image correction values ​​U11-U64 for all areas R11-R64 in the display area 220. After performing the image correction, the display control unit 32 sets the display position of the display content Iv to the display reference position P0.

[0083] After performing the vibration correction process and the image correction process, the display control unit 32 displays the display content Iv (S112).

[0084] When it is determined not to display the display content Iv (No in S103), the display control unit 32 makes the display content Iv non-displayable (S113).

[0085] The display control unit 32 judges whether or not to continue the display process (S114). For example, when the engine of the vehicle 200 is stopped, or when a button for instructing to end the display of the display content Iv is operated, the display control unit 32 ends the display process. If the display process is to be continued, the process returns to step S101.

[0086] 6. Operation of the correction processing device The operation of the correction control unit 52 of the vibration correction processing device 50 in the first embodiment will be described with reference to Fig. 10. Fig. 10 shows the calculation process of the vibration correction amount C performed by the correction control unit 52 of the vibration correction processing device 50.

[0087] The calculation process shown in Fig. 10 is started, for example, when the engine of the vehicle 200 is started, or when a button for instructing the start of display of the display content Iv is operated. The calculation process in Fig. 10 is started, for example, together with the display process in Fig. 7. Note that the calculation process shown in Fig. 10 may be started when a button for instructing the start of position correction of the display content Iv is operated.

[0088] The correction control unit 52 acquires content display information of the display content Iv from the display processing device 30 (S201). The correction control unit 52 acquires an attitude change amount indicating the angular velocity of the vehicle 200 output from the gyro sensor 41 (S202). The correction control unit 52 calculates the attitude of the vehicle 200, that is, the deviation amount y in the three axial directions, based on the attitude change amount (S203).

[0089] The correction control unit 52 calculates the vibration correction amount C based on the deviation amount y calculated based on the posture change amount (S204). For example, the correction control unit 52 determines the vibration correction amount C that returns the deviation amount y of the pitch angle to the original value.

[0090] The correction control unit 52 outputs information on the vibration correction amount C to the display processing device 30 (S205).

[0091] The correction control unit 52 judges whether or not to continue the calculation process (S206). For example, when the engine of the vehicle 200 is stopped, or when a button for instructing to end the display of the display content Iv is operated, the correction control unit 52 ends the calculation process. When the calculation process is to be continued, the process returns to step S201.

[0092] As described above, in this embodiment, the correction control unit 52 calculates the vibration correction amount C for the display position of the display content Iv based on the amount of change in posture.

[0093] 7. Effects and Supplements The display system 100 of the present disclosure includes a display processing device 30, an image correction unit 34, a posture detection device 40, a vibration correction processing device 50, and a process switching unit 32a. The display processing device 30 controls the display of the display content Iv. The image correction unit 34 performs an image correction process accompanied by a change in the display position of the display content Iv based on pre-stored image correction data 33g. The posture detection device 40 detects an amount of change in posture of a vehicle 200, which is a moving body. The vibration correction processing device 50 calculates a vibration correction amount C of the display position of the display content Iv based on the amount of change in posture of the moving body. The process switching unit 32a switches the order of the process between the image correction process accompanied by a change in the display position of the display content Iv and the vibration correction process that corrects the display position of the display content Iv based on the vibration correction amount C. The display processing device 30 controls the display position of the display content Iv by performing the image correction process and the vibration correction process based on the order of the processes switched by the process switching unit 32a.

[0094] With this configuration, it is possible to suppress misalignment of the display position of the display content Iv with respect to the superimposition target. In the display system 100, by switching the order of the image correction process and the vibration correction process, it is possible to suppress a decrease in the correction accuracy of the display position of the display content Iv.

[0095] The process switching unit 32a acquires the image processing time Td for the image correction process. The process switching unit 32a switches the order of the image correction process and the vibration correction process based on the image processing time Td.

[0096] With this configuration, the order of processing can be optimized according to the driving conditions, and the accuracy of both the vibration correction processing and the image correction processing can be improved to improve the superimposition performance and quality. The larger the content processing amount, the larger the image processing time Td. For example, when there is a lot of content to be displayed (for example, when the mobile body is running in an urban area), the image processing time Td becomes longer, and when there is a small amount of content to be displayed (for example, when the mobile body is running on a highway), the image processing time Td may become shorter. Therefore, by switching the order of the image correction processing and the vibration correction processing based on the image processing time Td, it is possible to perform optimal processing according to the driving conditions of the mobile body. For example, when the image processing time Td is large, the vibration correction processing is performed after the image correction processing, thereby shortening the delay time of the vibration correction processing and suppressing the vibration correction error. When the image processing time Td is small, the accuracy of the image correction processing can be improved by performing the image correction processing after the vibration correction processing.

[0097] In addition, even if the display content Iv is updated and the amount of information changes, the vibration correction process and the image correction process can be performed in an optimal processing order based on the image processing time Td. Specifically, if the display content Iv is updated and the amount of information changes, the image processing time Td also changes. For example, if the amount of information of the display content Iv increases, the image processing time Td becomes longer, and the delay time due to the vibration correction process may become longer. In this case, if the image correction process is performed after the vibration correction process, the vibration correction error becomes larger. In the display system 100, the order of the vibration correction process and the image correction process can be switched based on the image processing time Td, so that even if the display content Iv is updated and the amount of information changes, the deterioration of the correction accuracy of the display position of the display content Iv can be suppressed.

[0098] When the image processing time Td is equal to or less than the first threshold T1, the process switching unit 32a switches the process order so that the image correction process is performed after the vibration correction process. When the image processing time Td is greater than the first threshold T1, the process switching unit 32a switches the process order so that the image correction process is performed after the vibration correction process.

[0099] With this configuration, it is possible to easily determine whether to switch the order of the image correction process and the vibration correction process based on the image processing time Td and the first threshold value T1.

[0100] In the present embodiment, the display control unit 32 includes the process switching unit 32a. However, the present invention is not limited to this. For example, the process switching unit 32a does not have to be included in the display control unit 32.

[0101] In the present embodiment, an example has been described in which the process switching unit 32a switches the order of the image correction process and the vibration correction process based on the image processing time Td of the image correction process, but the present invention is not limited to this. For example, the process switching unit 32a may switch the order of the image correction process and the vibration correction process based on the running state of the moving object. Alternatively, the process switching unit 32a may switch the order of the delay time from correction to display or the increase amount thereof when the vibration correction process is performed after the image correction process. In this way, the process switching unit 32a may switch the order of the processes based on information other than the image processing time Td.

[0102] Second embodiment In the first embodiment, the order of the vibration correction process and the image correction process is switched based on the image processing time Td of the image correction process. In the present embodiment, the order of the vibration correction process and the image correction process is switched based on whether or not a specific frequency vibration is occurring.

[0103] The internal configuration of the display system 100A will be described with reference to Fig. 11. Fig. 11 is a block diagram showing the internal configuration of the display system 100A in the second embodiment. The display system 100A shown in Fig. 11 differs from the display system 100 of the first embodiment in that the correction control unit 52A includes a vibration detection unit 52c. The other configurations of the display system 100A shown in Fig. 11 are similar to those of the display system 100 of the first embodiment.

[0104] The vibration detection unit 52c detects whether or not the input signal includes a specific frequency band component. In this embodiment, the specific frequency band component is a high-frequency band component. Specifically, the vibration detection unit 52c detects whether or not the input signal includes a high-frequency band component equal to or higher than a predetermined frequency band. The vibration detection unit 52c includes a fluctuation amount calculation unit 52m, an offset removal filter 52e, a threshold crossing detection unit 52f, a counter 52g, and a vibration determination unit 52h.

[0105] The fluctuation amount calculation unit 52m calculates the amount of fluctuation of the vehicle 200 based on the amount of change in attitude output by the attitude detection device 40. For example, the fluctuation amount calculation unit 52m calculates the angle (pitch angle) around the pitch axis of the vehicle 200 by integrating the pitch angular velocity detected by the gyro sensor 41 as the amount of fluctuation. This makes it possible to detect the amount of fluctuation (pitch angle) of the vehicle 200 in the rotation direction around the Y axis (pitch axis) shown in FIG. 1 as the central axis. Note that, although the pitch angle is calculated in this embodiment, the yaw angle or roll angle may be calculated. For example, the angles around the X axis, the Y axis, and the Z axis may all be calculated. Note that the amount of deviation calculated by the deviation amount calculation unit 52a and the amount of deviation calculated by the fluctuation amount calculation unit 52m are both angle information, and therefore may be used in common or may be calculated as separate angle information.

[0106] The offset removal filter 52e removes the offset component of the input signal (variation amount), and as a result, attenuates the low-frequency band component of the input signal. The offset removal filter 52e is, for example, a high-pass filter. The relationship between the cut-off frequency Fc of the offset removal filter 52e and the lower limit F of the frequency band to be detected satisfies, for example, the following formula (1). Fc < F ≤ 1 / (2×T [sec]) ··· Formula (1) T is the total delay time (time lag) from the calculation of the correction amount of the virtual image to the display [sec]. However, the method of setting the frequency is not limited to formula (1) and may be adjusted as appropriate.

[0107] The threshold crossing detection unit 52f detects that the input value has crossed a predetermined threshold. Here, "detecting that the threshold has been crossed" means, for example, detecting that the input value has changed from less than the threshold to greater than or equal to the threshold, or from greater than or equal to the threshold to less than the threshold. When the threshold crossing detection unit 52f detects that the threshold has been crossed, it outputs a detection signal to the counter 52g. The threshold is, for example, zero or an arbitrary specific value. The arbitrary specific value is, for example, the value of the offset error.

[0108] The counter 52g measures the number of times the input value to the threshold crossing detection unit 52f has crossed a predetermined threshold. That is, the counter 52g counts the number of detection signals input from the threshold crossing detection unit 52f to the counter 52g.

[0109] The vibration determination unit 52h determines whether a specific frequency band component has occurred in the variation amount for a certain period of time or more based on the number of times measured by the counter 52g per unit time. The vibration determination unit 52h, for example, provides a time window and determines the occurrence of vibration having a specific frequency band component based on the change amount of the count value within a certain period of time. The determination result is sent from the vibration determination unit 52h to the correction amount calculation unit 52b. In this embodiment, the vibration determination unit 52h determines whether a high-frequency band component has occurred in the variation amount for a certain period of time or more based on the number of times measured by the counter 52g per unit time.

[0110] The operation of the display control unit 32 of the display processing device 30 will be described with reference to Fig. 12. Fig. 12 shows display processing performed by the display control unit 32 of the display processing device 30. Steps S301 to S305 and S308 to S314 in Fig. 12 are similar to steps S101 to S105 and S108 to S114 in Fig. 7 of the first embodiment.

[0111] The display control unit 32 acquires vehicle-related information including position information of the vehicle 200 from the information acquisition device 20 (S301). The display control unit 32 determines whether or not to display the display content Iv corresponding to the display target 230 based on the vehicle-related information (S302).

[0112] When the display control unit 32 determines to display the display content Iv (Yes in S303), it acquires information indicating the display reference position P0 of the display content Iv from the external device and acquires content information from the storage unit 33 (S304). The display control unit 32 outputs the content display information to the vibration correction processing device 50A (S305). For example, the content display information includes the display reference position and the content information.

[0113] The display control unit 32 acquires the occurrence detection result of vibration having high frequency band components (S306). The process switching unit 32a of the display control unit 32 acquires the occurrence detection result of vibration having high frequency band components from the vibration correction processing device 50A. Specifically, the display control unit 32 acquires a signal indicating that vibration of a specific frequency is occurring, which is output from the vibration correction processing device 50A. When the display control unit 32 receives the signal from the vibration correction processing device 50A, it recognizes that vibration of the specific frequency is occurring. When the display control unit 32 does not receive the signal from the vibration correction processing device 50A, it recognizes that vibration of the specific frequency is not occurring.

[0114] The process switching unit 32a switches the order of the image correction process and the vibration correction process based on the result of detection of the occurrence of vibration of a specific frequency. In this embodiment, the process switching unit 32a determines whether or not vibration having high frequency band components has occurred for a certain period of time or more (S307). The process switching unit 32a switches the order of the image correction process and the vibration correction process based on the determination of whether or not vibration having high frequency band components has occurred for a certain period of time or more.

[0115] If vibrations having high frequency band components have not occurred for a certain period of time or longer (No in S307), the process switching unit 32a switches the order of processes so that image correction process is performed after vibration correction process. That is, if it is not detected that vibrations having high frequency band components have occurred for a certain period of time or longer, the display processing device 30 first performs vibration correction process (S308). The display processing device 30 performs vibration correction process and then image correction process (S309).

[0116] When vibrations having high frequency band components have occurred for a certain period of time or more (Yes in S307), the process switching unit 32a switches the order of processes so that vibration correction process is performed after image correction process. That is, when it is detected that vibrations having high frequency band components have occurred for a certain period of time or more, the display processing device 30 performs image correction process (S310). The display processing device 30 performs image correction process and then vibration correction process (S311).

[0117] The vibration correction process may be the same as the vibration correction process of the first embodiment shown in Fig. 8. The image correction process may be the same as the image correction process of the first embodiment shown in Fig. 9.

[0118] After performing the vibration correction process and the image correction process, the display control unit 32 displays the display content Iv (S312).

[0119] When it is determined not to display the display content Iv (No in S303), the display control unit 32 makes the display content Iv non-display (S313).

[0120] The display control unit 32 judges whether or not to continue the display process (S314). If the display process is to be continued, the process returns to step S301.

[0121] The operation of the correction control unit 52A of the vibration correction processing device 50A in the second embodiment will be described with reference to Fig. 13. Fig. 13 shows the calculation process of the vibration correction amount C performed by the correction control unit 52A of the vibration correction processing device 50A in the second embodiment. Steps S401 to S403 and S406 to S408 in Fig. 13 are similar to steps S201 to S206 in Fig. 10 of the first embodiment.

[0122] The correction control unit 52A acquires content display information of the display content Iv from the display processing device 30 (S401). The correction control unit 52A acquires an attitude change amount indicating the angular velocity of the vehicle 200 output from the gyro sensor 41 (S402). The correction control unit 52A calculates the attitude of the vehicle 200, that is, the deviation amount y with respect to the three axial directions, based on the acquired attitude change amount (S403). Specifically, the deviation amount calculation unit 52a calculates the pitch angle of the vehicle 200 by integrating the angular velocity. The calculated deviation amount y is sent to the correction amount calculation unit 52b.

[0123] The fluctuation amount calculation unit 52m of the vibration detection unit 52c of the correction control unit 52A calculates the amount of fluctuation in the attitude of the vehicle 200, for example, the angle with respect to the pitch direction, based on the acquired amount of attitude change (S404). Specifically, the fluctuation amount calculation unit 52m calculates the pitch angle of the vehicle 200 by integrating the angular velocity. The calculated amount of fluctuation is sent to the offset removal filter 52e. Note that the amount of fluctuation may be calculated after the offset removal.

[0124] The offset removal filter 52e of the vibration detection unit 52c removes the offset component of the detected fluctuation amount, and as a result, the low frequency band component of the fluctuation amount is also attenuated. For example, when the fluctuation amount fluctuates as shown in Fig. 14A(a), the offset removal filter 52e removes the offset component OF1 included in the fluctuation amount, thereby obtaining the fluctuation amount that oscillates around 0 value as the center of amplitude as shown in Fig. 14A(b). The fluctuation amount from which the offset component OF1 has been removed and from which the low frequency band component has been attenuated is sent to the threshold crossing detection unit 52f.

[0125] The threshold crossing detector 52f detects that the inputted amount of change has crossed a predetermined threshold. For example, when the amount of change changes as shown in Fig. 14B(a), the threshold crossing detector 52f detects that the amount of change has changed from a threshold value of less than zero to a threshold value of zero or more, and outputs a detection signal to the counter 52g as shown in Fig. 14B(b). Note that the threshold crossing detector 52f may halve the detection cycle and count a decrease in the amount of change from a threshold value of less than zero to a threshold value of zero or more, in addition to an increase in the amount of change.

[0126] The counter 52g counts the number of detection signals input during a predetermined period of time. The predetermined period of time may be, for example, a time window or a recent predetermined time.

[0127] The vibration determination unit 52h determines whether or not vibration having a high-frequency band component is occurring in the amount of variation (S405). The vibration determination unit 52h determines whether or not the amount of variation includes a high-frequency band component by, for example, comparing the number of times the amount of variation crosses a threshold within a predetermined period with a count threshold. The vibration determination unit 52h compares the number of times counted by the counter 52g with the count threshold.

[0128] When the vibration determination unit 52h determines that the number of times counted by the counter 52g is smaller than the count threshold, it determines that the amount of fluctuation does not contain high-frequency band components for a certain period of time or more, and recognizes that the vibration of the vehicle 200 does not contain vibrations having high-frequency band components based on this determination result.

[0129] When the vibration determination unit 52h determines that the number of times counted by the counter 52g is equal to or greater than the count threshold, it recognizes that the amount of fluctuation includes vibration having a high frequency band component.

[0130] The correction control unit 52A outputs the occurrence detection result of the vibration having high frequency band components to the display processing device 30 (S405). Specifically, when the vibration having high frequency band components is occurring, a signal indicating that the vibration having high frequency band components is occurring is sent to the display processing device 30.

[0131] The correction control unit 52A calculates the vibration correction amount C based on the deviation amount y calculated based on the posture change amount (S407). For example, the correction control unit 52A determines the vibration correction amount C that returns the deviation amount y of the pitch angle to the original value.

[0132] The correction control unit 52A outputs information on the vibration correction amount C to the display processing device 30 (S408).

[0133] The correction control unit 52A judges whether or not to continue the calculation process (S409). For example, when the engine of the vehicle 200 is stopped, or when a button for instructing to end the display of the display content Iv is operated, the correction control unit 52A ends the calculation process. If the calculation process is to be continued, the process returns to step S401.

[0134] As described above, the display system 100A of this embodiment includes the vibration detection unit 52c that detects that vibrations having specific frequency band components occur for a certain period of time or longer in the posture change of the moving object. The process switching unit 32a switches the order of the image correction process and the vibration correction process based on the detection result of the vibration detection unit 52c.

[0135] With this configuration, it is possible to suppress misalignment of the display position of the display content Iv with respect to the superimposition target. In the display system 100A, by switching the order of the image correction process and the vibration correction process, it is possible to suppress a decrease in the correction accuracy of the display position of the display content Iv.

[0136] The vibration having a specific frequency band component is a high-frequency vibration having a high-frequency band component. When the vibration having the high-frequency band component occurs for a certain period of time or more, the process switching unit 32a switches the order of the processes so that the image correction process is performed after the image correction process when the vibration having the high-frequency band component has not occurred for a certain period of time or more. In addition, the process switching unit 32a switches the order of the processes so that the image correction process is performed after the image correction process when the vibration having the high-frequency band component has not occurred for a certain period of time or more.

[0137] With this configuration, the order of processing can be optimized according to the running conditions, so that the accuracy of the vibration correction processing and the image correction processing can be compatible to improve the superimposition performance and quality. The delay time of the vibration correction processing that can be tolerated for the target superimposition accuracy differs for each vibration frequency. The delay time of the vibration correction processing means the time from the calculation of the vibration correction amount to the display. For example, the higher the vibration frequency, the shorter the delay time of the vibration correction processing that can be tolerated to suppress the vibration correction error to a certain level or less. Therefore, by detecting that vibrations having high-frequency band components equal to or higher than a specific frequency band have occurred for a certain period of time or more in the attitude change of the moving body, and switching the order of the image correction processing and the vibration correction processing according to the detection result, the optimal processing can be performed according to the running conditions of the moving body. For example, if vibrations having high-frequency band components have occurred for a certain period of time or more, the delay time of the vibration correction processing can be shortened by performing the vibration correction processing after the image correction processing, thereby expanding the high-frequency band that can be corrected, and suppressing the vibration correction error. If vibrations having high-frequency band components have not occurred for a certain period of time or more, the accuracy of the image correction processing can be improved by performing the image correction processing after the vibration correction processing.

[0138] Furthermore, similarly to the first embodiment, in this embodiment too, even if the display content Iv is updated and the amount of information changes, vibration correction processing and image correction processing can be performed in the optimal processing order based on the detection results of vibration having high-frequency band components.

[0139] In addition, since the frequency band and amplitude of the vibrations generated vary depending on the road conditions, when applying filter processing such as a high-pass filter, a low-pass filter, or a band-pass filter, the amplitude of the vibration must be included in the judgment criteria, but it is difficult to predict the amplitude of the vibration. It is difficult to set an optimal threshold value for the filter processing. In contrast, according to this embodiment, when detecting high-frequency components using a counter value, it is not necessary to include the amplitude of the vibration in the judgment criteria, and the high-frequency band can be detected with high accuracy.

[0140] In the present embodiment, the vibration detection unit 52c includes the fluctuation amount calculation unit 52m, the offset removal filter 52e, the threshold cross detection unit 52f, the counter 52g, and the vibration determination unit 52h. However, the vibration detection unit 52c is not limited to this. It is sufficient for the vibration detection unit 52c to detect that a vibration having a specific frequency band component has occurred for a certain period of time or more.

[0141] In the present embodiment, an example has been described in which, when vibrations having high-frequency band components are occurring, image correction processing is performed and then vibration correction processing is performed, but the present invention is not limited to this. For example, when vibrations having high-frequency band components are occurring, vibration correction processing may not be performed. Alternatively, the vibration correction amount C may be set to "0".

[0142] In the present embodiment, an example has been described in which the specific frequency vibration is vibration having a high frequency band component, but the present invention is not limited thereto. For example, the specific frequency vibration may be vibration having a low frequency band component below a predetermined frequency band. The process switching unit 32a may switch the order of the processes between the vibration correction process and the image correction process based on whether or not the vibration having the low frequency band component has been generated for a certain period of time or more. Specifically, when the vibration having the low frequency band component has been generated for a certain period of time or more, the process switching unit 32a may perform the vibration correction process and then the image correction process. When the vibration having the low frequency band component has not been generated for a certain period of time or more, the process switching unit 32a may perform the vibration correction process and then the image correction process.

[0143] Third embodiment In the first and second embodiments, the vibration correction amount C is calculated based on the amount of deviation, regardless of the order of processing. In this embodiment, when image correction processing is performed after vibration correction processing, the vibration correction amount C is calculated based on the amount of deviation, and when vibration correction processing is performed after image correction processing, the vibration correction amount C is calculated based on the amount of deviation and the image correction error. Also, in this embodiment, the calculation process of the vibration correction amount C is switched based on the image processing time Td.

[0144] The internal configuration of a display system 100B will be described with reference to Fig. 15. Fig. 15 is a block diagram showing the internal configuration of a display system 100B in the third embodiment. The display system 100B shown in Fig. 15 differs from the display system 100 of the first embodiment in that a correction control unit 52B includes an image correction error calculation unit 52d. The other configurations of the display system 100B shown in Fig. 15 are similar to those of the display system 100 of the first embodiment.

[0145] The image correction error calculation unit 52d calculates an image correction error of the image correction process caused by the vibration correction process that corrects the display shift caused by the posture change of the vehicle 200. Specifically, the image correction error calculation unit 52d calculates the initial position of the display content Iv based on the content display information output from the display processing device 30. The initial position means the display position of the display content Iv when the vibration correction amount is "0". For example, the initial position may be the display reference position initially acquired from an external device, or may be the position of a characteristic part of the display content Iv when the vibration correction amount is "0".

[0146] In this embodiment, the initial position is the display position of the characteristic part of the display content Iv when the vibration correction amount is "0". For example, if the display content Iv is an arrow, the characteristic part is the tip of the arrow. In this case, the initial position is the display position of the tip of the arrow. FIG. 16A is a diagram showing an example of the initial position x1 of the display content Iv. The display content Iv shown in FIG. 16A is an arrow having a linear shape extending in one direction. In the example shown in FIG. 16A, the initial position x1 of the display content Iv coincides with the display reference position P0, which is the display position of the display content Iv when the vibration correction amount is "0". FIG. 16B is a diagram showing another example of the initial position x1 of the display content Iv. The display content Iv shown in FIG. 16B is an arrow having a shape bent by 90°. In the example shown in FIG. 16B, the initial position x1 of the display content Iv is different from the display reference position P0.

[0147] The image correction error calculation unit 52d acquires an image correction value (image correction data) from the display processing device 30 via the communication unit 31. The image correction error calculation unit 52d calculates an image correction error based on the initial position of the display content Iv, the amount of deviation calculated by the deviation amount calculation unit 52a, and the image correction value (image correction data) acquired from the display processing device 30.

[0148] For example, the image correction error calculation unit 52d obtains a correction value (first image correction value) for image correction at the initial position of the display content Iv. Also, the image correction error calculation unit 52d obtains a correction value (second image correction value) for image correction at the display position of the display content Iv that has been vibration corrected based on the displacement amount calculated by the displacement amount calculation unit 52a. The image correction error calculation unit 52d calculates the image correction error by subtracting the first image correction value from the second image correction value. That is, the image correction error is the difference between the first image correction value at the initial position of the display content Iv and the second image correction value at the display position of the display content Iv that is displayed when vibration correction based on the displacement amount calculated by the displacement amount calculation unit 52a is performed.

[0149] In this embodiment, the image correction error calculation unit 52d stores a correction value table in which image correction values ​​corresponding to deviation amounts from the initial position are assigned. For example, the image correction error calculation unit 52d has a storage unit and stores the correction value table in the storage unit. The image correction error calculation unit 52d calculates the image correction error based on the initial position, the deviation amount, and the correction value table.

[0150] In this specification, the correction value of the image correction corresponding to the amount of deviation from the initial position may be referred to as a first correction value, and the correction value table may be referred to as a first correction value table.

[0151] When image correction processing is performed after vibration correction processing, the correction amount calculation unit 52b calculates the vibration correction amount C based on the shift amount y. In this embodiment, when the image processing time Td is equal to or less than the first threshold value T1, the correction amount calculation unit 52b calculates the vibration correction amount C based on the shift amount y.

[0152] When vibration correction processing is performed after image correction processing, the correction amount calculation unit 52b calculates a vibration correction amount C based on the shift amount y and an image correction error of the image correction processing caused by the vibration correction processing that corrects the display shift caused by the posture change of the vehicle 200. In this embodiment, when the image processing time Td is greater than the first threshold value T1, the correction amount calculation unit 52b calculates the vibration correction amount C based on the shift amount y and the image correction error. Specifically, the correction amount calculation unit 52b calculates the vibration correction amount C based on the shift amount y calculated by the shift amount calculation unit 52a and the image correction error calculated by the image correction error calculation unit 52d.

[0153] With reference to Fig. 17, an image correction error E1 of the image correction process caused by the vibration correction process that corrects the display shift caused by the posture change of the vehicle 200 will be described. Fig. 17 shows an example in which an image correction error E1 of the image correction process occurs when the vibration correction process is performed after the image correction process. In the example shown in Fig. 17, a distortion occurs in the vertical direction of the display area 220. Therefore, when the display area 220 is divided into a plurality of divided areas, the sizes of the plurality of divided areas differ due to the distortion.

[0154] 17, in the vertical direction of the display area 220, the sizes of the upper areas L21, L22 and the lower areas L25, L26 are smaller than the divided areas when there is no distortion. Also, the sizes of the central areas L23, L24 are larger than the divided areas when there is no distortion. Therefore, in the upper areas L21, L22 and the lower areas L25, L26, the display content Iv is corrected to be larger in the vertical direction. Also, in the central areas L23, L24, the display content Iv is corrected to be smaller in the vertical direction.

[0155] That is, in the image correction process of "A1 → A2" in the central region L23 of Fig. 17, a reduction process is applied, and the display content Iv is corrected so as to be smaller in the vertical direction. In the image correction process of "A4 → A3" in the upper region L22 of Fig. 17, an enlargement process is applied, and the display content Iv is corrected so as to be larger in the vertical direction.

[0156] First, a case will be described where the display content Iv is displayed in the central region L23 before the correction process, and the vibration correction amount by the vibration correction process is 0. In this case, in the image correction process, a reduction process from "A1 to A2" in the central region L23 in Fig. 17 is applied, and the display content Iv is corrected so as to be smaller in the vertical direction.

[0157] In the vibration correction process, since the vibration correction amount is "0", the display position of the display content Iv is not changed by the vibration correction process. Therefore, the display position of the display content Iv21 after vibration correction is not changed from the display reference position P0. That is, the display content Iv21 does not shift from the superimposed position. Note that in the example shown in FIG. 17, the display content Iv is an arrow having a shape bent by 90 degrees, and the initial position x1 in the yaw axis direction (Z axis direction) is the same as the display reference position P0, but is not limited to this. The initial position x1 may be different from the display reference position P0.

[0158] Next, a case will be described where the display content Iv is displayed in the central region L23 before the correction process, and the vibration correction amount in the yaw axis direction (Z axis direction) by the vibration correction process is "C10". In this case, the target display position P2 of the display content Iv22 after the correction is calculated by "display reference position (initial position) P0 + vibration correction amount C10". For example, if the initial position (display reference position P0) of the display content Iv when the vibration correction amount is "0" is x1, the target display position P2 is calculated by "initial position x1 + vibration correction amount C10". In the example shown in FIG. 17, the display content Iv22 is displayed in the upper region L22, which is one region above the central region L23 (see "A3" in FIG. 17).

[0159] When the display content Iv22 is displayed in the upper region L22, it is preferable that the image correction process be an enlargement process of "A4 → A3" in the upper region L22 of Fig. 17. That is, when the display content Iv22 is displayed in the upper region L22, it is preferable that the display content Iv is corrected so as to be larger in the vertical direction.

[0160] However, as described above, in the correction process, the vibration correction process is performed after the image correction process. In the example shown in Fig. 17, before the vibration correction process is performed, that is, when the image correction process is performed, the display content Iv is displayed in the central region L23. Therefore, in the image correction process, a reduction process of "A1 → A2" in the central region L23 of Fig. 17 is applied instead of the enlargement process of "A4 → A3" in the upper region L22 of Fig. 17. That is, the display content Iv is corrected so as to become smaller in the vertical direction.

[0161] In this way, when the display position of the display content Iv22 is changed by the vibration correction process, an image correction error E1 occurs in the image correction process performed before the vibration correction process, and as a result, the corrected display content Iv22 is displayed at a display position P3 that is lower than the target display position P2.

[0162] Therefore, in the display system 100B of the present embodiment, when the vibration correction process is performed after the image correction process, the vibration correction amount C of the display position of the display content Iv is calculated based on the vibration and the image correction error. This makes it possible to prevent the display position of the display content Iv after the correction from shifting from the target display position, and to prevent a decrease in the correction accuracy of the display position of the display content Iv.

[0163] The operation of the correction control unit 52B of the vibration correction processing device 50B in the third embodiment will be described with reference to Fig. 18. Fig. 18 shows the correction process performed by the correction control unit 52B of the vibration correction processing device 50B. Steps S501 to S503, S506, and S509 to S510 in Fig. 18 are similar to steps S201 to S206 in Fig. 10 of the first embodiment.

[0164] The correction control unit 52B acquires content display information of the display content Iv from the display processing device 30 (S501). The correction control unit 52B acquires an attitude change amount indicating the angular velocity of the vehicle 200 output from the gyro sensor 41 (S502). The correction control unit 52B calculates the attitude of the vehicle 200, that is, the deviation amount y in the three axial directions, based on the attitude change amount (S503).

[0165] The correction control unit 52B changes the calculation process of the vibration correction amount C based on the order of the image correction process and the vibration correction process. Specifically, when the image correction process is performed after the vibration correction process, the correction control unit 52B calculates the vibration correction amount based on the deviation amount y. When the vibration correction process is performed after the image correction process, the correction control unit 52B calculates the vibration correction amount C based on the deviation amount y and the image correction error E1. In this embodiment, the correction control unit 52B switches the calculation process of the vibration correction amount C based on the image processing time Td of the image correction process and the first threshold value T1.

[0166] The correction control unit 52B acquires information on the image processing time Td (S504). For example, the correction control unit 52B acquires information on the image processing time Td from the display processing device 30.

[0167] The correction control unit 52B determines whether the image processing time Td is equal to or less than the first threshold T1 (S505). If the image processing time Td is equal to or less than the first threshold T1 (Yes in S505), the correction control unit 52B calculates the vibration correction amount C based on the deviation amount y (S506).

[0168] When the image processing time Td is greater than the first threshold T1 (No in S504), the correction control unit 52B calculates the vibration correction amount C based on the shift amount y and the image correction error E1. Specifically, the correction control unit 52B calculates the image correction error E1 of the image correction caused by the vibration correction process that corrects the display shift caused by the posture change of the vehicle 200 based on the content display information, the shift amount y calculated based on the posture change amount, and the image correction value U (image correction data) (S507). The correction control unit 52B calculates the initial position x1, which is the display position of the display content Iv when the vibration correction amount is "0", based on the content display information output from the display processing device 30. The initial position x1 may change depending on the display reference position P0 and / or the characteristic part of the display content Iv (see FIG. 16A and FIG. 16B). Therefore, the initial position x1 is calculated based on the content display information (for example, the display reference position P0, the content information). The correction control unit 52B calculates an image correction error E1 based on the initial position x1 of the display content Iv, the shift amount y, and an image correction value U (image correction data). The image correction error E1 is calculated as a difference between a first image correction value at the initial position x1 of the display content Iv and a second image correction value at the display position of the display content Iv after vibration correction based on the shift amount y.

[0169] The calculation of the image correction error E1 will be described. The image correction error E1 can be calculated for all the divided areas shown in FIG. 6A and FIG. 6B, while there is only one vibration correction amount C. Therefore, there is also only one image correction error E1 that can be added to the vibration correction. Since the display content Iv is displayed across multiple divided areas, the correction accuracy of the display content Iv can be improved by selecting which divided area among the multiple divided areas in which the display content Iv is located, the image correction error of which is to be added to the vibration correction amount. In this embodiment, the divided area used to calculate the image correction error E1 is selected based on the display position of the characteristic part of the display content Iv, that is, the initial position x1. This can improve the accuracy of the display deviation correction of the characteristic part of the display content Iv.

[0170] An example of calculation of image correction error E1 will be described with reference to Fig. 19A and Fig. 19B. Fig. 19A shows an example of the display position of display content Iv before and after vibration correction in display area 220. Fig. 19B is a diagram for explaining a first image correction value before vibration correction and a second image correction value after vibration correction in the example of Fig. 19A. In Fig. 19A, display content Iv shown by a solid line indicates the display position before vibration correction, and display content Iv shown by a dotted line indicates the display position after vibration correction. Fig. 19B shows an image correction table including image correction values ​​U11 to U64 assigned to each of a plurality of divided areas R11 to R64 of display area 220 shown in Fig. 19A.

[0171] In the example shown in Fig. 19A, before vibration correction, i.e., when the vibration correction amount is "0", the tip of the arrow, which is a characteristic part of the display content Iv, is located in the first divided area R33. Therefore, the initial position x1 of the display content Iv becomes the first divided area R33. After vibration correction, the tip of the arrow, which is a characteristic part of the display content Iv, becomes the second divided area R13 above the first divided area R33. Therefore, the display position x2 of the characteristic part of the display content Iv after vibration correction becomes the second divided area R13.

[0172] 19B, the first image correction value assigned to the first divided region R33 is "U33," and the second image correction value assigned to the second divided region R13 is "U13."

[0173] Therefore, the image correction error E1 is calculated by subtracting the first image correction value U33 from the second image correction value U13.

[0174] In this embodiment, the correction control unit 52B has a first correction value table to which a first correction value Ea of image correction corresponding to a deviation amount y from the initial position x1 is assigned. The correction control unit 52B calculates an image correction error E1 based on the initial position x1, the deviation amount y, and the first correction value table. Specifically, the correction control unit 52B acquires a correction value Ex as a first image correction value of image correction at the initial position x1. For example, the correction value Ex is acquired from image correction values ​​U11 to U64 assigned to the multiple divided regions R11 to R64 shown in FIG. 6A and FIG. 6B. In addition, the correction control unit 52B refers to the first correction value table and acquires a first correction value Ea as a second image correction value of image correction at the display position x2 of the display content Iv after vibration correction based on the deviation amount y from the initial position x1. The correction control unit 52B calculates the image correction error E1 by subtracting the correction value Ex from the first correction value Ea.

[0175] Fig. 20 shows an example of a first correction value table including a first correction value Ea of image correction assigned to a deviation amount y from an initial position x1 of a display content Iv. The example shown in Fig. 20 shows a first correction value table including a first correction value Ea indicating an image correction value in the Z-axis direction corresponding to an angle deviation amount y around the Y-axis from the initial position x1. As shown in Fig. 20, in the first correction value table, a first correction value Ea is defined every time the angle deviation amount y changes by 0.01 [deg.]. For example, when the angle deviation amount y is -0.01≦y<0.00, the first correction value Ea is E100, and when the angle deviation amount y is -M / 100≦y<-(M-1) / 100, the first correction value Ea is Em. When the angle deviation amount y is 0.00≦y<+0.01, the first correction value Ea is E200, and when the angle deviation amount y is +(N-1) / 100≦y<+N / 100, the first correction value Ea is En. Note that "M" and "N" are arbitrary constants.

[0176] In the example shown in Fig. 20, the first correction value Ea is assigned every time the deviation amount y from the initial position x1 changes by 0.01 [deg.], but the present invention is not limited to this. The range of the deviation amount y to which the first correction value Ea is assigned is not limited to every 0.01 [deg.]. In the example shown in Fig. 20, the first correction value Ea is assigned to the deviation amount y in the pitch axis direction from the initial position x1, but a correction value may be assigned to the deviation amount in the yaw axis direction and the roll direction from the initial position x1.

[0177] An example of calculation of the image correction error E1 using the first correction value table shown in FIG. 20 will be described. The correction value Ex (first image correction value) of the image correction at the initial position x1 of the display content Iv is calculated based on the initial position x1 and the image correction data 33g. For example, when the initial position x1 is within the partitioned region R23 shown in FIG. 6A, the correction value Ex is determined to be the image correction value U23. The first correction value Ea (second image correction value) of the image correction at the display position x2 of the display content Iv after the vibration correction based on the deviation amount y from the initial position x1 is calculated based on the initial position x1, the deviation amount y, and the first correction value table. For example, when the display position x2 of the display content Iv after the vibration correction is at a position of -0.015 [deg.] around the Y axis from the initial position x1, the first correction value table shown in FIG. 20 is referred to, and the image correction value E101 in the Z direction corresponding to the deviation amount "-0.02≦y<-0.01" is determined as the first correction value Ea. In this case, the image correction error E1 is calculated by subtracting the image correction value U23 from the image correction value E101.

[0178] In this embodiment, the correction control unit 52B calculates an initial position x1, which is the display position of the characteristic part of the display content Iv when the vibration correction amount is "0", based on the content display information. For example, when the display content Iv is an arrow, the correction control unit 52B calculates the display position of the tip of the arrow when the vibration correction amount is "0" as the initial position x1. The first correction value Ea is calculated based on the difference between a first image correction value (correction value Ex) of a first display position (initial position) of the characteristic part of the display content Iv when the vibration correction amount is "0" and a second image correction value (first correction value Ea) of a second display position of the characteristic part of the display content Iv after vibration correction. This allows the display processing device 30 to control the display position of the characteristic part of the display content Iv.

[0179] Returning to FIG. 18, after calculating the image correction error E1, the correction control unit 52B calculates the vibration correction amount C based on the deviation amount y and the image correction error E1 (S508). For example, the correction control unit 52B determines the vibration correction amount C that restores the pitch angle deviation amount y and eliminates the image correction error E1. Specifically, the correction control unit 52B converts the deviation amount y, which is angle information, into the number of pixels to calculate the deviation correction amount. The correction control unit 52B adds the image correction error E1 to the deviation correction amount to determine the vibration correction amount C that restores the number of pixels by the amount of deviation.

[0180] The correction control unit 52B outputs information on the vibration correction amount C to the display processing device 30 (S509).

[0181] The correction control unit 52B judges whether or not to continue the calculation process (S510). If the calculation process is to be continued, the process returns to step S501.

[0182] As described above, in this embodiment, when the process switching unit 32a switches the order of processes so that the vibration correction process is performed after the image correction process, the vibration correction processing device 50B calculates the vibration correction amount C based on the shift amount y and the image correction error E1. Specifically, when the image processing time Td is greater than the first threshold value T1, the vibration correction processing device 50B calculates the shift amount y of the display position of the display content Iv based on the posture change amount of the moving body. The vibration correction processing device 50B calculates the initial position x1 of the display content Iv based on the content display information output from the display processing device 30. The vibration correction processing device 50B calculates the image correction error E1 based on the initial position x1, the shift amount y, and the image correction value U (image correction data). The vibration correction processing device 50B calculates the vibration correction amount C based on the shift amount y and the image correction error E1.

[0183] With this configuration, it is possible to suppress the positional deviation of the display position of the display content Iv relative to the superimposition target. Specifically, when the vibration correction process is performed after the image correction process, it is possible to suppress the positional deviation of the display position of the display content Iv caused by the image correction error E1 of the image correction caused by the vibration correction process. This makes it possible to suppress the deterioration of the correction accuracy of the display position of the display content Iv.

[0184] The vibration correction processing device 50B has a first correction value table in which a first correction value Ea of the image correction processing corresponding to the deviation amount y from the initial position x1 is assigned, and calculates an image correction error E1 based on the initial position x1, the deviation amount y, and the first correction value table.

[0185] With this configuration, the image correction error E1 can be easily calculated based on the deviation amount y from the initial position x1, which makes it possible to further reduce the position deviation of the display position of the display content Iv.

[0186] In the present embodiment, an example of switching the calculation process of the vibration correction amount C based on the image processing time Td has been described, but the present invention is not limited thereto. For example, the vibration correction processing device 50B may acquire a process switching signal from the process switching unit 32a of the display processing device 30 and switch the calculation process of the vibration correction amount C based on the process switching signal. In this case, when the image processing time Td is greater than the first threshold value T1, the display processing device 30 may output a signal to the vibration correction processing device 50B to switch to the calculation process of the vibration correction amount C based on the deviation amount y and the image correction error E1. In addition, when the image processing time Td is equal to or less than the first threshold value T1, the display processing device 30 may output a signal to the vibration correction processing device 50B to switch to the calculation process of the vibration correction amount C based on the deviation amount y.

[0187] In the present embodiment, an example has been described in which the correction control unit 52B has a first correction value table indicating the first correction value Ea of the image correction corresponding to the deviation amount y from the initial position of the display content Iv, but the present invention is not limited to this. For example, the correction control unit 52B may not have a first correction value table. In this case, the storage unit 53 may store the first correction value table. The correction control unit 52B may read out the first correction value table stored in the storage unit 53 and calculate the image correction error E1.

[0188] In the present embodiment, an example has been described in which the image correction error E1 is calculated using a first correction value table in which a first correction value Ea of image correction corresponding to an angular deviation amount y from an initial position x1 of the display content Iv is assigned, but the present invention is not limited to this. For example, the image correction error E1 may be calculated using a second correction value table in which a second correction value of image correction corresponding to a deviation correction amount in number of pixels from the initial position of the display content Iv is assigned.

[0189] Fig. 21 shows a flowchart illustrating the calculation process of the vibration correction amount C in a modified example of the third embodiment. The calculation process of the vibration correction amount C in the modified example shown in Fig. 21 includes steps S520 to S522 instead of steps S507 to S508 of the third embodiment shown in Fig. 18. Steps S520 to S522 show the calculation process of the vibration correction amount C in the case where the vibration correction process is performed after the image correction process. The process of steps S520 to S522 in Fig. 21 will be described below.

[0190] The correction control unit 52B calculates the amount of misalignment correction G based on the amount of misalignment y calculated based on the amount of posture change (S520). The correction control unit 52B converts the amount of misalignment y into the number of pixels, and determines the amount of misalignment correction G that offsets the amount of misalignment indicated by the number of pixels.

[0191] The correction control unit 52B calculates an image correction error E1 based on the content display information and the misalignment correction amount G (S521). The correction control unit 52B has a second correction value table in which a second correction value Eb of image correction corresponding to the misalignment correction amount G from the initial position x1 of the display content Iv is assigned. For example, the second correction value table is stored in a storage unit provided in the image correction error calculation unit 52d.

[0192] FIG. 22 shows an example of a second correction value table including a second correction value Eb of image correction assigned to a misalignment correction amount G from an initial position x1 of the display content Iv. In the example shown in FIG. 22, a second correction value table including a second correction value Eb indicating an image correction value in the Z-axis direction corresponding to a misalignment correction amount G in the Z direction from the initial position x1 is shown. As shown in FIG. 22, in the second correction value table, a second correction value Eb is defined for each misalignment correction amount G of 1 [pixel]. For example, when the misalignment correction amount G is -1.0≦G<0.0, the second correction value Eb is E300, and when the misalignment correction amount G is -J≦G<-(J-1), the second correction value Eb is Ej. Also, when the misalignment correction amount G is 0.0≦G<+1.0, the second correction value Eb is E400, and when the misalignment correction amount G is +(K-1)≦G<+K, the second correction value Eb is Ek. Note that "J" and "K" represent arbitrary constants.

[0193] In the example shown in Fig. 22, the second correction value Eb is assigned to the deviation correction amount G every time the deviation correction amount G from the initial position x1 changes by 1 [pixel], but the present invention is not limited to this. The range of the deviation correction amount G to which the second correction value Eb is assigned is not limited to 1 [pixel]. In the example shown in Fig. 22, the second correction value Eb is assigned to the deviation correction amount G in the pitch axis direction, but the second correction value may be assigned to the deviation correction amount in the yaw axis direction and the roll direction.

[0194] The correction control unit 52B calculates an image correction error E1 based on the initial position x1 of the display content Iv, the deviation correction amount G, and the second correction value table. Specifically, the correction control unit 52B acquires a correction value Ex (first image correction value) of image correction at the initial position x1. Furthermore, the correction control unit 52B refers to the second correction value table and acquires a second correction value Eb (second image correction value) of image correction at the display position of the display content Iv after vibration correction based on the deviation correction amount G from the initial position x1. The correction control unit 52B calculates the image correction error E1 by subtracting the correction value Ex from the second correction value Eb.

[0195] An example of calculation of the image correction error E1 using the second correction value table shown in FIG. 22 will be described. The image correction error E1 is calculated by the difference between the correction value Ex (first image correction value) at the initial position x1 of the display content Iv and the second correction value Eb (second image correction value) at the display position x2 of the display content Iv after the vibration correction based on the deviation correction amount G from the initial position x1. The correction value Ex of the image correction at the initial position x1 of the display content Iv is calculated based on the initial position x1 and the image correction data 33g. For example, when the initial position x1 is within the partitioned region R23 shown in FIG. 6A, the correction value Ex is determined to be the image correction value U23 shown in FIG. 6B. The second correction value Eb of the image correction at the display position of the display content Iv after the vibration correction based on the deviation correction amount G is calculated based on the initial position x1, the deviation correction amount G, and the second correction value table. For example, when the display position of the display content Iv after the vibration correction based on the misalignment correction amount G is −0.8 [pixel] from the initial position x1 in the Z-axis direction, the image correction value E300 in the Z direction corresponding to the misalignment correction amount “−1.0≦G<0.0” is determined as the second correction value Eb by referring to the second correction value table shown in Fig. 22. In this case, the image correction error E1 is calculated by subtracting the image correction value U23 from the image correction value E300.

[0196] Returning to FIG. 21, the correction control unit 52B calculates the vibration correction amount C based on the deviation correction amount G and the image correction error E1 (S522). The correction control unit 52B calculates the vibration correction amount C by adding the image correction error E1 to the deviation correction amount G. Specifically, the correction control unit 52B adds the image correction error E1 to the deviation correction amount G and determines the vibration correction amount C that restores the number of pixels by the amount of deviation. For example, the correction control unit 52B determines the vibration correction amount C that restores the deviation correction amount G of the pitch angle and eliminates the image correction error E1.

[0197] Even with this configuration, it is possible to suppress misalignment of the display position of the display content Iv relative to the superimposition target. Specifically, when vibration correction processing is performed after image correction processing, it is possible to suppress misalignment of the display position of the display content Iv due to image correction error E1 of image correction caused by the vibration correction processing. This makes it possible to suppress deterioration in the correction accuracy of the display position of the display content Iv.

[0198] (Fourth embodiment) In the third embodiment, the calculation process of the vibration correction amount C is switched based on the image processing time Td. In this embodiment, the calculation process of the vibration correction amount C is switched based on whether or not a specific frequency vibration is occurring.

[0199] The internal configuration of a display system 100C will be described with reference to Fig. 23. Fig. 23 is a block diagram showing the internal configuration of a display system 100C in the fourth embodiment. The display system 100C shown in Fig. 23 has the same configuration as the display system 100A in the second embodiment, except that a correction control unit 52C includes an image correction error calculation unit 52d. Moreover, the image correction error calculation unit 52d in the fourth embodiment is the same as that in the third embodiment.

[0200] When performing image correction processing after vibration correction processing, the correction amount calculation unit 52b calculates the vibration correction amount C based on the deviation amount y. In this embodiment, the specific frequency vibration is a vibration having a high frequency band component. When a vibration having a high frequency band component has not occurred for a certain period of time or more, the correction amount calculation unit 52b calculates the vibration correction amount C based on the deviation amount y.

[0201] When vibration correction processing is performed after image correction processing, the correction amount calculation unit 52b calculates a vibration correction amount C based on the shift amount y and an image correction error of the image correction processing caused by the vibration correction processing that corrects the display shift caused by the posture change of the vehicle 200. In this embodiment, when vibration having high frequency band components occurs for a certain period of time or more, the correction amount calculation unit 52b calculates the vibration correction amount C based on the shift amount y and the image correction error. Specifically, the correction amount calculation unit 52b calculates the vibration correction amount C based on the shift amount y calculated by the shift amount calculation unit 52a and the image correction error calculated by the image correction error calculation unit 52d.

[0202] The operation of the correction control unit 52C of the vibration correction processing device 50C in the fourth embodiment will be described with reference to Fig. 24. Fig. 24 shows the correction process performed by the correction control unit 52C of the vibration correction processing device 50C. Steps S601 to S603 and S607 to S611 in Fig. 24 are similar to steps S501 to S503 and S506 to S510 in Fig. 18 of the third embodiment. Also, steps S604 and S605 in Fig. 24 are similar to steps S404 and S405 in Fig. 13 of the second embodiment.

[0203] The correction control unit 52C acquires content display information of the display content Iv from the display processing device 30 (S601). The correction control unit 52C acquires an attitude change amount indicating the angular velocity of the vehicle 200 output from the gyro sensor 41 (S602). The correction control unit 52C calculates the attitude of the vehicle 200, that is, the deviation amount y in the three axial directions, based on the attitude change amount (S603). The correction control unit 52C detects whether or not vibration having high-frequency band components occurs for a certain period of time or more in the deviation amount y (S604). The correction control unit 52C outputs the detection result of the vibration having high-frequency band components to the display processing device 30 (S605).

[0204] The correction control unit 52C switches the calculation process of the vibration correction amount C based on the detection result of the vibration having the high frequency band component. Specifically, the correction control unit 52C determines whether or not the vibration having the high frequency band component is occurring (S606). When the vibration having the high frequency band component is not occurring (No in S606), the correction control unit 52C calculates the vibration correction amount C based on the deviation amount y (S607).

[0205] When vibration having high frequency band components is occurring (Yes in S606), the correction control unit 52C calculates the vibration correction amount C based on the displacement amount y and the image correction error E1. Specifically, the correction control unit 52C calculates the image correction error E1 based on the content display information, the displacement amount y, and the image correction value U (image correction data) (S608). After calculating the image correction error E1, the correction control unit 52C calculates the vibration correction amount C based on the displacement amount y and the image correction error E1 (S609).

[0206] The correction control unit 52C outputs information on the vibration correction amount C to the display processing device 30 (S610).

[0207] The correction control unit 52C determines whether or not to continue the calculation process (S611). If the calculation process is to be continued, the process returns to step S601.

[0208] As described above, in this embodiment, when the process switching unit 32a switches the order of processes so that the vibration correction process is performed after the image correction process, the vibration correction processing device 50B calculates the vibration correction amount C based on the deviation amount y and the image correction error E1. Specifically, when the specific frequency vibration occurs for a certain period of time or more, the vibration correction processing device 50B calculates the deviation amount y of the display position of the display content Iv based on the posture change amount of the moving body. The vibration correction processing device 50B calculates the initial position x1 of the display content Iv based on the content display information output from the display processing device 30. The vibration correction processing device 50B calculates the image correction error E1 based on the initial position x1, the deviation amount y, and the image correction value U (image correction data). The vibration correction processing device 50B calculates the vibration correction amount C based on the deviation amount y and the image correction error E1.

[0209] With this configuration, it is possible to suppress misalignment of the display position of the display content Iv with respect to the superimposition target. For example, when vibration correction processing is performed after image correction processing is performed when vibration having high frequency band components occurs for a certain period of time or more, it is possible to suppress misalignment of the display position of the display content Iv due to image correction error E1 of the image correction caused by the vibration correction processing. This makes it possible to suppress deterioration in the correction accuracy of the display position of the display content Iv.

[0210] In the present embodiment, when the process switching unit 32a switches the process order so that the vibration correction process is performed after the image correction process, the vibration correction processing device 50B calculates the vibration correction amount C based on the deviation amount y and the image correction error E1, but the present invention is not limited to this. For example, the vibration correction processing device 50B may calculate the vibration correction amount C based on the deviation correction amount G based on the deviation amount y and the image correction error E1.

[0211] Fig. 25 shows a flowchart illustrating the calculation process of the vibration correction amount C in the modified example of the fourth embodiment. The calculation process of the vibration correction amount C in the modified example shown in Fig. 25 includes steps S620 to S622 instead of steps S608 to S609 in the fourth embodiment shown in Fig. 22. Steps S620 to S622 show the calculation process of the vibration correction amount C in the case where the vibration correction process is performed after the image correction process. Note that steps S620 to S622 are similar to steps S520 to S522 in the modified example of the third embodiment shown in Fig. 21, and therefore a description thereof will be omitted.

[0212] Even with this configuration, it is possible to suppress misalignment of the display position of the display content Iv relative to the superimposition target. Specifically, when vibration correction processing is performed after image correction processing, it is possible to suppress misalignment of the display position of the display content Iv due to image correction error E1 of image correction caused by the vibration correction processing. This makes it possible to suppress deterioration in the correction accuracy of the display position of the display content Iv.

[0213] In the present embodiment, an example has been described in which the specific frequency vibration is vibration having a high frequency band component, but the present invention is not limited thereto. For example, the specific frequency vibration may be vibration having a low frequency band component below a predetermined frequency band. The process switching unit 32a may switch the order of the processes between the vibration correction process and the image correction process based on whether or not the vibration having the low frequency band component has been generated for a certain period of time or more. Specifically, when the vibration having the low frequency band component has been generated for a certain period of time or more, the process switching unit 32a may perform the vibration correction process and then the image correction process. When the vibration having the low frequency band component has not been generated for a certain period of time or more, the process switching unit 32a may perform the vibration correction process and then the image correction process.

[0214] Fifth embodiment In the first to fourth embodiments, the order of correction processes and the calculation process of vibration correction amount C are switched based on the image processing time Td or the detection result of vibration having high frequency band components equal to or higher than a specific frequency band. In this embodiment, the order of correction processes and the calculation process of vibration correction amount C are switched based on the first image correction errors assigned to each of the multiple divided areas of the display area.

[0215] The internal configuration of the display system of the fifth embodiment is similar to the internal configuration of the display system 100B of the third embodiment shown in Fig. 15. Therefore, the fifth embodiment will be described using the same reference numerals as in the third embodiment.

[0216] The operation of the display control unit 32 of the display processing device 30 in the fifth embodiment will be described with reference to Fig. 26. Fig. 26 shows display processing performed by the display control unit 32 of the display processing device 30 in the fifth embodiment. Steps S701 to S705 and S708 to S714 in Fig. 26 are similar to steps S101 to S105 and S108 to S114 in Fig. 7 of the first embodiment.

[0217] The display control unit 32 acquires vehicle-related information including position information of the vehicle 200 from the information acquisition device 20 (S701). The display control unit 32 determines whether or not to display the display content Iv corresponding to the display target 230 based on the vehicle-related information (S702).

[0218] When it is determined that the display content Iv is to be displayed (Yes in S703), the display control unit 32 acquires information indicating the display reference position P0 of the display content Iv from the external device and acquires content information from the storage unit 33 (S704). The display control unit 32 outputs the content display information to the vibration correction processing device 50B (S705).

[0219] The display control unit 32 acquires a first image correction error E11 (S706). The first image correction error E11 is an image correction error that occurs due to a change in the display position caused by the vibration correction process after the image correction that involves a change in the display position of the display content Iv. In this embodiment, the first image correction error E11 is an error that takes into account the image correction errors of all the divided areas R11 to R64 in the display area 220 of the display content Iv.

[0220] The first image correction error E11 is calculated in a manner similar to that used for calculating the image correction error E1 described in the third embodiment.

[0221] For example, the first image correction error E11 is calculated based on the initial position x1 of the display content Iv and the shift amount y using the first correction value table shown in Fig. 20. Alternatively, the first image correction error E11 is calculated based on the initial position x1 of the display content Iv and the shift correction amount G using the second correction value table shown in Fig. 22.

[0222] The first image correction error E11 is calculated by the vibration correction processing device 50B and output to the display processing device 30. The display processing device 30 acquires the first image correction error E11 from the vibration correction processing device 50B.

[0223] The process switching unit 32a of the display control unit 32 switches the order of the image correction process and the vibration correction process based on the first image correction error E11.

[0224] The process switching unit 32a judges whether the absolute value of the first image correction error E11 is equal to or greater than the second threshold T2 (S707). If the absolute value of the first image correction error E11 is equal to or greater than the second threshold T2 (Yes in S707), the display control unit 32 switches the order of processes so that the image correction process is performed after the vibration correction process. That is, if the absolute value of the first image correction error E11 is equal to or greater than the second threshold T2, the display processing device 30 first performs the vibration correction process (S708). The display processing device 30 performs the vibration correction process and then the image correction process (S709).

[0225] When the absolute value of the first image correction error E11 is smaller than the second threshold T2 (No in S707), the process switching unit 32a switches the order of processes so that the image correction process is performed after the vibration correction process. That is, when the absolute value of the first image correction error E11 is smaller than the second threshold T2, the display processing device 30 performs the image correction process (S710). The display processing device 30 performs the image correction process and then the vibration correction process (S711).

[0226] After performing the vibration correction process and the image correction process, the display control unit 32 displays the display content Iv (S712).

[0227] When it is determined not to display the display content Iv (No in S703), the display control unit 32 makes the display content Iv non-display (S713).

[0228] The display control unit 32 judges whether or not to continue the display process (S714). If the display process is to be continued, the process returns to step S701.

[0229] The operation of the correction control unit 52B of the vibration correction processing device 50B in the fifth embodiment will be described with reference to Fig. 27. Fig. 27 shows the calculation process of the vibration correction amount C performed by the correction control unit 52B of the vibration correction processing device 50B in the fifth embodiment. Steps S801 to S803 and S807 to S811 in Fig. 27 are similar to steps S501 to S503 and S506 to S510 in Fig. 18 of the third embodiment.

[0230] The correction control unit 52B acquires content display information of the display content Iv from the display processing device 30 (S801). The correction control unit 52B acquires an attitude change amount indicating the angular velocity of the vehicle 200 output from the gyro sensor 41 (S802). The correction control unit 52B calculates the attitude of the vehicle 200, that is, the deviation amount y in the three axial directions, based on the attitude change amount (S803).

[0231] The correction control unit 52B switches the calculation process of the vibration correction amount C based on the first image correction error E11 and the second threshold value T2.

[0232] The correction control unit 52B calculates the first image correction error E11 (S804). For example, similar to the third embodiment, the correction control unit 52B calculates the first image correction error E11 based on the initial position x1 of the display content Iv and the shift amount y using the first correction value table shown in Fig. 20. Alternatively, the correction control unit 52B calculates the first image correction error E11 based on the initial position x1 of the display content Iv and the shift correction amount G using the second correction value table shown in Fig. 22.

[0233] The correction control unit 52B outputs the first image correction error E11 to the display processing device 30 (S805).

[0234] The correction control unit 52B determines whether the absolute value of the first image correction error E11 is equal to or greater than the second threshold T2 (S806). If the absolute value of the first image correction error E11 is equal to or greater than the second threshold T2 (Yes in S806), the correction control unit 52B calculates the vibration correction amount C based on the deviation amount y (S807).

[0235] When the absolute value of the first image correction error E11 is smaller than the second threshold T2 (No in S806), the correction control unit 52B calculates the vibration correction amount C based on the shift amount y and the second image correction error E12. The second image correction error E12 is an image correction error of the image correction caused by the vibration correction process that corrects the display shift caused by the posture change of the vehicle 200. The second image correction error E12 corresponds to the image correction error E1 in the first to fourth embodiments. In this embodiment, the second image correction error E12 is an image correction error of the characteristic part of the display content Iv. The correction control unit 52B calculates the second image correction error E12 of the image correction caused by the vibration correction process that corrects the display shift caused by the posture change of the vehicle 200 based on the content display information, the shift amount y, and the image correction value U (image correction data) (S808). The calculation of the second image correction error E12 is similar to the calculation of the image correction error E1 in the third embodiment. After calculating the second image correction error E12, the correction control unit 52B calculates the vibration correction amount C based on the deviation amount y and the second image correction error E12 (S809).

[0236] The correction control unit 52B outputs information on the vibration correction amount C to the display processing device 30 (S810).

[0237] The correction control unit 52B judges whether or not to continue the calculation process (S811). If the calculation process is to be continued, the process returns to step S801.

[0238] As described above, in this embodiment, the process switching unit 32a switches the order of the image correction process and the vibration correction process based on the first image correction error E11 that occurs due to a change in the display position due to the vibration correction process after the image correction process, which involves a change in the display position of the display content Iv.

[0239] With this configuration, it is possible to suppress misalignment of the display position of the display content Iv relative to the superimposition target. In this embodiment, the order of the vibration correction process and the image correction process can be switched based on the first image correction error E11, so that the optimal process can be performed according to the traveling conditions of the moving object. For example, when the first image correction error E11 is small, the vibration correction process is performed after the image correction process, thereby shortening the delay time of the vibration correction process and suppressing the vibration correction error. When the first image correction error E11 is large, the accuracy of the image correction process can be improved by performing the image correction process after the vibration correction process.

[0240] Furthermore, even when the display content Iv is updated and the display range changes, it is possible to suppress a decrease in the correction accuracy of the display position of the display content Iv.

[0241] In addition, in this embodiment, the calculation process of the vibration correction amount C is switched based on the first image correction error E11 caused by the change in the display position by the vibration correction process after the image correction process accompanied by the change in the display position of the display content Iv. Specifically, when the absolute value of the first image correction error E11 is equal to or greater than the second threshold T2, the vibration correction processing device 50B calculates the vibration correction amount C based on the shift amount y. When the absolute value of the first image correction error E11 is smaller than the second threshold T2, the vibration correction processing device 50B calculates the vibration correction amount C based on the shift amount y and the second image correction error E12. With this configuration, it is possible to further suppress the positional deviation of the display position of the display content Iv with respect to the superimposition target. Specifically, when the vibration correction process is performed after the image correction process, it is possible to suppress the positional deviation of the display position of the display content Iv caused by the second image correction error E12 of the image correction caused by the vibration correction process. This makes it possible to further suppress the deterioration of the correction accuracy of the display position of the display content Iv.

[0242] In this embodiment, the first image correction error E11 is an error taking into consideration image correction errors of all partitioned regions R11-R64 in the display region 220 of the display content Iv, and the second image correction error E12 is an image correction error of the characteristic portion of the display content Iv. In this embodiment, the quality of the entire display region 220 (for example, the magnitude of the image correction error) is determined by the first image correction error E11, i.e., the minimum value or average value of all partitioned regions R11-R64, and the second image correction error E12 is added to suppress display deviation of the characteristic portion of the display content Iv.

[0243] In the present embodiment, the display processing device 30 has been described as acquiring the first image correction error E11 from the vibration correction processing device 50 as an example, but is not limited thereto. For example, the display processing device 30 may acquire the amount of deviation y from the vibration correction processing device 50, and calculate the first image correction error E11 based on the initial position x1 of the display content Iv and the amount of deviation y using the first correction value table. Alternatively, the display processing device 30 may acquire the amount of deviation correction G from the vibration correction processing device 50, and calculate the first image correction error E11 based on the initial position x1 of the display content Iv and the amount of deviation correction G using the second correction value table.

[0244] Furthermore, when the first image correction error E11 is calculated by the display processing device 30, the vibration correction processing device 50B may obtain the first image correction error E11 from the display processing device 30.

[0245] In the present embodiment, the process switching unit 32a switches the process order based on the first image correction error E11 and the second threshold T2, but the present invention is not limited to this. For example, the process switching unit 32a may switch the process order based on the minimum or average value of the first image correction error E11 and the second threshold T2.

[0246] In the present embodiment, an example of calculating the first image correction error E11 and the second image correction error E12 has been described, but the present invention is not limited to this. For example, when emphasis is placed on suppressing the display shift of the characteristic part of the display content Iv, the first image correction error E11 may be the same as the second image correction error E12.

[0247] In the present embodiment, when the process switching unit 32a switches the process order so that the vibration correction process is performed after the image correction process, the vibration correction processing device 50B calculates the vibration correction amount C based on the deviation amount y and the second image correction error E12, but the present invention is not limited to this. For example, the vibration correction processing device 50B may calculate the vibration correction amount C based on the deviation correction amount G based on the deviation amount y and the second image correction error E12.

[0248] Fig. 28 shows a flowchart illustrating the calculation process of the vibration correction amount C in the modified example of the fifth embodiment. The calculation process of the vibration correction amount C in the modified example shown in Fig. 28 includes steps S820 to S822 instead of steps S808 to S809 in the fifth embodiment shown in Fig. 27. Steps S820 to S822 show the calculation process of the vibration correction amount C in the case where the vibration correction process is performed after the image correction process. Note that steps S820 to S822 are similar to steps S520 to S522 in the modified example of the third embodiment shown in Fig. 21, and therefore a description thereof will be omitted.

[0249] Even with this configuration, it is possible to suppress misalignment of the display position of the display content Iv relative to the superimposition target. Specifically, when vibration correction processing is performed after image correction processing, it is possible to suppress misalignment of the display position of the display content Iv due to the second image correction error E12 of the image correction caused by the vibration correction processing. This makes it possible to suppress a decrease in the correction accuracy of the display position of the display content Iv.

[0250] (Other embodiments) As described above, the above embodiment has been described as an example of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to this, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are appropriately performed. Therefore, other embodiments will be exemplified below.

[0251] In the above embodiment, the projection device 10, the information acquisition device 20, the display processing device 30, the attitude detection device 40, and the vibration correction processing devices 50 to 50C are each separate devices. However, a plurality of devices may be integrally formed as one device. For example, the display processing device 30 and the vibration correction processing devices 50 to 50C may be integrally formed as one device. The information acquisition device 20 and the display processing device 30 may be integrally formed as one device. The attitude detection device 40 and the vibration correction processing devices 50 to 50C may be integrally formed as one device. The devices formed separately are connected to each other so that they can communicate with each other by wire or wirelessly. Note that the projection device 10, the information acquisition device 20, the display processing device 30, the attitude detection device 40, and the vibration correction processing devices 50 to 50C may all be formed as one device. In this case, the communication units 31 and 51 may not be required.

[0252] In the above embodiment, an example in which the information acquisition device 20 includes the GPS module 21 has been described. However, the information acquisition device 20 may include a distance sensor that measures the distance and direction from the vehicle 200 to a surrounding object, and may output distance information indicating the measured distance and direction to the display processing device 30. The information acquisition device 20 may include a vehicle speed sensor that detects the speed of the vehicle 200, or may include a navigation system. The information acquisition device 20 may include one or more of the GPS module 21, a distance sensor, a camera, an image processing device, an acceleration sensor, a radar, a sound wave sensor, and a white line detection device of an ADAS (Advanced Driver-Assistance Systems). In this case, the GPS module 21, the distance sensor, the camera, and the like having the function of the information acquisition device 20 may be built into one device, or may be individually attached to the vehicle 200.

[0253] In the above embodiment, an example in which the attitude detection device 40 includes the gyro sensor 41 has been described. However, the attitude detection device 40 may include an acceleration sensor that detects the acceleration of the vehicle 200, and may output the detected acceleration as the attitude change amount. The attitude detection device 40 may include a vehicle height sensor that detects the height from the road surface, and may output the detected height as the attitude change amount. The attitude detection device 40 may include other known sensors. The attitude detection device 40 may include one or more of the gyro sensor 41, an acceleration sensor, and a vehicle speed sensor. In this case, the gyro sensor 41, the acceleration sensor, and the vehicle height sensor that function as the attitude detection device 40 may be built into one device, or may be individually attached to the vehicle 200.

[0254] In the above embodiment, an example in which the image correction unit 34 is included in the display processing device 30 has been described, but the present invention is not limited to this. For example, the image correction unit 34 may be included in the vibration correction processing devices 50 to 50C. Alternatively, the image correction unit 34 may be included in a device other than the display processing device 30 and the vibration correction processing devices 50 to 50C.

[0255] In the above embodiment, an example in which the process switching unit 32a is included in the display control unit 32 has been described, but the present invention is not limited to this. The process switching unit 32a does not have to be included in the display control unit 32. For example, the process switching unit 32a may be included in the vibration correction processing devices 50 to 50C. Alternatively, the process switching unit 32a may be included in another device other than the display processing device 30 and the vibration correction processing devices 50 to 50C.

[0256] In the above embodiment, an example has been described in which the image correction unit 34 is included in the display processing device 30, but the present invention is not limited to this example. The image correction unit 34 may be included in the vibration correction processing devices 50 to 50C.

[0257] In the above embodiment, the moving body is a vehicle 200 such as an automobile. However, the moving body is not limited to the vehicle 200. The moving body may be a vehicle that runs on the ground, for example, a train or a motorcycle. The moving body may be an unmanned vehicle that can run autonomously.

[0258] In the above embodiment, the image is displayed in front of the moving object. However, the position where the image is displayed is not limited to the front. For example, the image may be displayed to the side or rear of the moving object.

[0259] In the above embodiment, an example has been described in which the display systems 100 to 100C are HUD systems. However, the display systems 100 to 100C do not have to be HUD systems. The display systems 100 to 100C may include a liquid crystal display or an organic EL display instead of the projection device 10. The display systems 100 to 100C may include a screen and a projector.

[0260] (Overview of the embodiment) (1) The display system of the present disclosure includes a display processing device that controls the display of display content, an image correction unit that performs image correction processing involving a change in the display position of the display content based on pre-stored image correction data, an attitude detection device that detects an amount of change in attitude of a moving body, a vibration correction processing device that calculates a vibration correction amount for the display position of the display content based on the amount of change in attitude of the moving body, and a process switching unit that switches the order of the image correction processing involving a change in the display position of the display content and the vibration correction processing that corrects the display position of the display content based on the vibration correction amount, and the display processing device controls the display position of the display content by performing the image correction processing and the vibration correction processing based on the process order switched by the process switching unit, thereby making it possible to suppress misalignment of the display position of the display content.

[0261] (2) In the display system of (1), the process switching unit may obtain an image processing time for the image correction process, and switch the order of the image correction process and the vibration correction process based on the image processing time.

[0262] (3) In the display system of (2), the process switching unit may switch the process order so that, when the image processing time is equal to or less than a first threshold, the image correction process is performed after the vibration correction process, and when the image processing time is greater than the first threshold, the process switching unit may switch the process order so that the image correction process is performed after the vibration correction process.

[0263] (4) In the display system of (1), the display system may further include a vibration detection unit that detects whether vibrations having specific frequency band components occur for a certain period of time or more in the posture change of the moving body, and the processing switching unit may switch the order of processing between the image correction processing and the vibration correction processing based on the detection result of the vibration detection unit.

[0264] (5) In the display system of (4), the vibration having a specific frequency band component is a high-frequency vibration having a high-frequency band component, and the processing switching unit may switch the processing order so that, when the high-frequency vibration has occurred for a certain period of time or more, an image correction processing is performed followed by a vibration correction processing, and when the high-frequency vibration has not occurred for a certain period of time or more, the processing order may be switched so that the vibration correction processing is performed followed by an image correction processing.

[0265] (6) In the display system of (1), the process switching unit may acquire a first image correction error that occurs due to a change in the display position caused by a vibration correction process after an image correction process that involves a change in the display position of the display content, and switch the order of the image correction process and the vibration correction process based on the first image correction error.

[0266] (7) In the display system of (6), the process switching unit may switch the process order so that, when the absolute value of the first image correction error is greater than or equal to the second threshold, the vibration correction process is performed after the image correction process, and may switch the process order so that, when the absolute value of the first image correction error is smaller than the second threshold, the vibration correction process is performed after the image correction process.

[0267] (8) In the display systems of (1) to (7), when the processing switching unit switches the order of processing so that vibration correction processing is performed after image correction processing, the vibration correction processing device may calculate an amount of shift of the display position of the display content based on an amount of change in posture of the moving body, calculate an initial position of the display content based on content display information output from the display processing device, calculate a second image correction error of the image correction processing caused by the vibration correction processing based on the initial position, the amount of shift, and the image correction data, and calculate an amount of vibration correction based on the amount of shift and the second image correction error.

[0268] In the display system of (9)(8), the vibration correction processing device may have a first correction value table in which a first correction value of the image correction processing corresponding to the amount of deviation from the initial position is assigned, and a second image correction error may be calculated based on the initial position, the amount of deviation, and the first correction value table.

[0269] (10) In the display systems of (1) to (7), the vibration correction processing device may calculate an amount of deviation of a display position of the display content based on an amount of change in posture of the moving body, calculate an amount of deviation correction based on the amount of deviation, calculate an initial position of the display content based on content display information output from the display processing device, calculate a second image correction error of the image correction processing caused by the vibration correction processing based on the initial position, the amount of deviation correction, and the image correction data, and calculate the vibration correction amount based on the amount of deviation correction and the second image correction error.

[0270] (11) In the display system of (10), the vibration correction processing device may have a second correction value table in which a second correction value of the image correction processing corresponding to a deviation correction amount from the initial position is assigned, and a second image correction error may be calculated based on the initial position, the deviation correction amount, and the second correction value table.

[0271] (12) In the display system of (1) to (11), the display processing device may control a display position of at least a characteristic portion that determines a shape of the display content.

[0272] (13) In the display system of (1) to (12), the image correction unit may perform distortion correction on the display content.

[0273] (14) The display system according to any one of (1) to (13) may further include a projection device that projects light representing the display content.

[0274] (15) In any of the display systems (1) to (14), the moving body may be a vehicle, and the display content may be content displayed in front of a windshield of the vehicle. [Industrial Applicability]

[0275] The present disclosure is applicable to a display system that controls the display position of a display content in accordance with the movement of a moving object. [Explanation of symbols]

[0276] 10 Projection device 20 Information acquisition device 21 GPS Module 30 Display processing device 31 Communications Department 32 Display control section 33 Storage section 34 Image correction section 40 Attitude detection device 41 Gyro sensor 50 Vibration correction processing device 51 Communications Department 52 Correction control section 52a Displacement amount calculation unit 52b Correction amount calculation section 52c Vibration detection unit 52d Image correction error calculation unit 52e Offset Removal Filter 52f Threshold crossing detection unit 52g Counter 52h Vibration judgment section 52m Fluctuation calculation section 53 Storage section 100 Display System 200 vehicles 210 Windshield 220 display area

Claims

1. a display processing device that controls display of display content; an image correction unit that performs an image correction process involving changing a display position of the display content based on pre-stored image correction data; an attitude detection device for detecting an attitude change amount of a moving object; a vibration correction processing device that calculates a vibration correction amount for a display position of the display content based on an amount of change in posture of the moving object; a process switching unit that switches the order of an image correction process involving a change in a display position of the display content and a vibration correction process that corrects the display position of the display content based on the vibration correction amount; Equipped with The process switching unit is Acquire an image processing time of the image correction processing; when the image processing time is equal to or less than a first threshold, switching the order of the processes so that the image correction process is performed after the vibration correction process; when the image processing time is greater than a first threshold value, switching the order of the processes so that the image correction process is performed after the vibration correction process; the display processing device controls a display position of the display content by performing the image correction process and the vibration correction process based on the order of the processes switched by the process switching unit. Display system.

2. A display processing device for controlling display of display content; an image correction unit that performs an image correction process involving changing a display position of the display content based on pre-stored image correction data; an attitude detection device for detecting an attitude change amount of a moving object; a vibration correction processing device that calculates a vibration correction amount for a display position of the display content based on an amount of change in posture of the moving object; a process switching unit that switches the order of an image correction process involving a change in a display position of the display content and a vibration correction process that corrects the display position of the display content based on the vibration correction amount; Equipped with The process switching unit is obtain a first image correction error caused by a change in the display position due to a vibration correction process after an image correction process involving a change in the display position of the display content; when the absolute value of the first image correction error is equal to or greater than a second threshold value, switching the order of the processes so that the image correction process is performed after the vibration correction process is performed; when the absolute value of the first image correction error is smaller than a second threshold value, switching the order of the processes so that the image correction process is performed after the vibration correction process; the display processing device controls a display position of the display content by performing the image correction process and the vibration correction process based on the order of the processes switched by the process switching unit. Display system.

3. When the process switching unit switches the process order so that the vibration correction process is performed after the image correction process, The vibration correction processing device includes: calculating a shift amount of a display position of the display content based on an amount of change in posture of the moving object; Calculating an initial position of the display content based on content display information output from the display processing device; calculating a second image correction error of the image correction process caused by the vibration correction process based on the initial position, the deviation amount, and the image correction data; calculating the vibration correction amount based on the deviation amount and the second image correction error; 3. A display system according to claim 1 or 2.

4. The vibration correction processing device includes: a first correction value table in which a first correction value of the image correction process corresponding to the amount of deviation from the initial position is assigned; calculating the second image correction error based on the initial position, the deviation amount, and the first correction value table; The display system of claim 3.

5. The vibration correction processing device includes: calculating a shift amount of a display position of the display content based on an amount of change in posture of the moving object; Calculating a deviation correction amount based on the deviation amount; Calculating an initial position of the display content based on content display information output from the display processing device; calculating a second image correction error of the image correction process caused by the vibration correction process based on the initial position, the deviation correction amount, and the image correction data; calculating the vibration correction amount based on the deviation correction amount and the second image correction error; 3. A display system according to claim 1 or 2.

6. The vibration correction processing device includes: a second correction value table in which a second correction value of the image correction process corresponding to the deviation correction amount from the initial position is assigned; calculating the second image correction error based on the initial position, the deviation correction amount, and the second correction value table; The display system of claim 5.

7. A display processing device for controlling display of display content; an image correction unit that performs an image correction process involving changing a display position of the display content based on pre-stored image correction data; an attitude detection device for detecting an attitude change amount of a moving object; a vibration correction processing device that calculates a vibration correction amount for a display position of the display content based on an amount of change in posture of the moving object; a process switching unit that switches the order of an image correction process involving a change in a display position of the display content and a vibration correction process that corrects the display position of the display content based on the vibration correction amount; Equipped with the display processing device controls a display position of the display content by performing the image correction process and the vibration correction process based on the process order switched by the process switching unit; When the process switching unit switches the process order so that the vibration correction process is performed after the image correction process, The vibration correction processing device includes: calculating a shift amount of a display position of the display content based on an amount of change in posture of the moving object; Calculating an initial position of the display content based on content display information output from the display processing device; calculating a second image correction error of the image correction process caused by the vibration correction process based on the initial position, the deviation amount, and the image correction data; calculating the vibration correction amount based on the deviation amount and the second image correction error; Display system.

8. The vibration correction processing device comprises: a first correction value table in which a first correction value of the image correction process corresponding to the amount of deviation from the initial position is assigned; calculating the second image correction error based on the initial position, the deviation amount, and the first correction value table; 8. The display system of claim 7.

9. A display processing device for controlling display of display content; an image correction unit that performs an image correction process involving changing a display position of the display content based on pre-stored image correction data; an attitude detection device for detecting an attitude change amount of a moving object; a vibration correction processing device that calculates a vibration correction amount for a display position of the display content based on an amount of change in posture of the moving object; a process switching unit that switches the order of an image correction process involving a change in a display position of the display content and a vibration correction process that corrects the display position of the display content based on the vibration correction amount; Equipped with the display processing device controls a display position of the display content by performing the image correction process and the vibration correction process based on the process order switched by the process switching unit; The vibration correction processing device includes: calculating a shift amount of a display position of the display content based on an amount of change in posture of the moving object; Calculating a deviation correction amount based on the deviation amount; Calculating an initial position of the display content based on content display information output from the display processing device; calculating a second image correction error of the image correction process caused by the vibration correction process based on the initial position, the deviation correction amount, and the image correction data; calculating the vibration correction amount based on the deviation correction amount and the second image correction error; Display system.

10. The vibration correction processing device comprises: a second correction value table in which a second correction value of the image correction process corresponding to the deviation correction amount from the initial position is assigned; calculating the second image correction error based on the initial position, the deviation correction amount, and the second correction value table; 10. The display system of claim 9.

11. A vibration detection unit is further provided which detects whether a vibration having a specific frequency band component occurs for a certain period of time or more in a posture change of the moving body, the process switching unit switches an order of the image correction process and the vibration correction process based on a detection result of the vibration detection unit. A display system according to any one of claims 7 to 10.

12. the display processing device controls a display position of at least a characteristic portion that determines a shape of the display content; A display system according to any one of claims 1 to 11.

13. The image correction unit performs distortion correction of the display content. A display system according to any one of claims 1 to 12.

14. Further comprising a projection device for projecting light representing the display content. A display system according to any one of claims 1 to 13.

15. the moving object is a vehicle, The display content is content to be displayed in front of a windshield of a vehicle. A display system according to any one of claims 1 to 14.

Images