Drawing system, display system, display control system, drawing method, and program

Abstract

To improve the accuracy of correcting displacement of display position of content relating to an object in a display image.SOLUTION: A drawing system 100 includes a first acquisition unit 11, a second acquisition unit 12, a drawing unit 13, an estimation unit 14, and a correction unit 15. The first acquisition unit 11 acquires position information indicating a position of an object from a detection system 400. The second acquisition unit 12 acquires move information indicating a moving state of a mobile body. The drawing unit 13 draws a display image including content related to the object. The estimation unit 14 estimates a delay period by comparing the position information with estimated position information indicating a position of the object estimated based on the move information of the mobile body. The correction unit 15 corrects displacement of the display position of the content in the display image due to the moving mobile body, in the delay period.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a rendering system, a display system, a display control system, a rendering method, and a program that render a display image that is projected onto a display medium so that an occupant of a moving body can view it as a virtual image. [Background technology]

[0002] Patent Document 1 discloses a head-up display device that projects a display image onto a display medium to allow a passenger in a moving vehicle to visually recognize a virtual image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6775188 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a drawing system and the like that can improve the accuracy of correcting a shift in the display position of content related to an object in a displayed image. [Means for solving the problem]

[0005] A rendering system according to one aspect of the present disclosure includes a first acquisition unit, a second acquisition unit, a rendering unit, an estimation unit, and a correction unit. The first acquisition unit acquires position information indicating the position of an object from a detection system that detects the object around a moving body. The second acquisition unit acquires movement information indicating a movement state of the moving body. The rendering unit renders a display image, based on the position information and the movement information, that is projected onto a display medium to be viewed as a virtual image by an occupant of the moving body and that includes content related to the object. The estimation unit compares the position information with estimated position information indicating the position of the object estimated based on the movement information of the moving body, to estimate a delay period including a delay from when the detection system detects the object to when the first acquisition unit acquires the position information. The correction unit corrects a shift in the display position of the content in the display image during the delay period due to movement of the moving body.

[0006] A display system according to one aspect of the present disclosure includes the rendering system and a projection unit, wherein the projection unit projects the display image rendered by the rendering system onto the display medium so that the display image is visually recognized as a virtual image by a passenger of the vehicle.

[0007] A display control system according to one aspect of the present disclosure includes a first acquisition unit, a second acquisition unit, an estimation unit, and a correction unit. The first acquisition unit acquires position information indicating the position of an object from a detection system that detects the object around a moving body. The second acquisition unit acquires movement information indicating a movement state of the moving body. The estimation unit compares the position information with estimated position information indicating the position of the object estimated based on the movement information of the moving body, thereby estimating a delay period including a delay from when the detection system detects the object to when the first acquisition unit acquires the position information. The correction unit corrects a shift in the display position of the content in a display image caused by movement of the moving body during the delay period. The display image is projected onto a display medium based on the position information and the movement information so that the display image can be viewed as a virtual image by a passenger of the moving body, and includes content related to the object.

[0008] A display control system according to one aspect of the present disclosure includes a display control unit that controls display of a display image. The display image is drawn based on position information indicating the position of an object acquired from a detection system that detects an object around a moving body and movement information indicating a movement state of the moving body, and is projected onto a display medium to be viewed as a virtual image by an occupant of the moving body, and includes content related to the object. The display control unit controls display of the display image so that a corrected display image is projected onto the display medium. The corrected display image is an image in which a shift in the display position of the content in the display image due to movement of the moving body during a delay period has been corrected. The delay period is a period including a delay between when the detection system detects the object and when the position information is acquired, the delay being estimated by comparing the position information with estimated position information indicating the position of the object estimated based on the movement information of the moving body.

[0009] A rendering method according to one aspect of the present disclosure includes a first acquisition step, a second acquisition step, a rendering step, an estimation step, and a correction step. In the first acquisition step, position information indicating the position of an object around a moving body is acquired from a detection system that detects the object. In the second acquisition step, movement information indicating a movement state of the moving body is acquired. In the rendering step, a display image including content related to the object is rendered based on the position information and the movement information, the display image being projected onto a display medium so that an occupant of the moving body can view the image as a virtual image. In the estimation step, a delay period including a delay from when the object is detected by the detection system to when the position information is acquired in the first acquisition step is estimated by comparing the position information with estimated position information indicating the position of the object estimated based on the movement information of the moving body. In the correction step, a shift in the display position of the content in the display image during the delay period due to movement of the moving body is corrected.

[0010] A program according to one aspect of the present disclosure causes one or more processors to execute the rendering method. [Effects of the Invention]

[0011] The drawing system and the like of the present disclosure can improve the accuracy of correcting the deviation of the display position of content related to an object in a display image. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing an outline of a display system having a drawing system according to the first embodiment. [Figure 2] FIG. 2 is a diagram showing an example of use of the display system according to the first embodiment. [Figure 3] FIG. 3 is a diagram showing an example of a predetermined area onto which an image displayed by the display system according to the first embodiment is projected. [Figure 4] FIG. 4 is an explanatory diagram of the correction process performed by the correction unit of the drawing system according to the first embodiment. [Figure 5] FIG. 5 is a graph for explaining the correction process performed by the correction unit of the drawing system according to the first embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of the operation of the drawing system according to the first embodiment. [Figure 7] FIG. 7 is a graph for explaining an example of the operation of the drawing system according to the first embodiment. [Figure 8] FIG. 8 is a flowchart showing an example of the operation of the drawing system according to the second embodiment. [Figure 9] FIG. 9 is a flowchart illustrating an example of a correction process performed by the correction unit of the drawing system according to the second embodiment. [Figure 10] FIG. 10 is a block diagram showing an outline of a display system having a drawing system according to the first modified example. [Figure 11] FIG. 11 is a block diagram showing an outline of a display system having a drawing system according to the second modified example. [Figure 12] FIG. 12 is a flowchart showing an example of the operation of the drawing system in the second modified example. [Figure 13] FIG. 13 is a diagram illustrating the advantages of the drawing system in the second modified example. [Figure 14] FIG. 14 is a diagram illustrating another advantage of the drawing system according to the second modified example. [Figure 15] FIG. 15 is an explanatory diagram of another example of the operation of the drawing system in the second modified example. [Figure 16] FIG. 16 is an explanatory diagram of yet another example of the operation of the drawing system in the second modified example. [Figure 17] FIG. 17 is a block diagram showing an outline of a display system having a display control system in the third modified example. [Figure 18] FIG. 18 is a block diagram showing an outline of another example of a display system having a display control system in the third modified example. [Figure 19] FIG. 19 is a block diagram showing an outline of a display system having a display control system according to the fourth modified example. DETAILED DESCRIPTION OF THE INVENTION

[0013] A rendering system according to one aspect of the present disclosure includes a first acquisition unit, a second acquisition unit, a rendering unit, an estimation unit, and a correction unit. The first acquisition unit acquires position information indicating the position of an object from a detection system that detects the object around a moving body. The second acquisition unit acquires movement information indicating a movement state of the moving body. The rendering unit renders a display image, based on the position information and the movement information, that is projected onto a display medium to be viewed as a virtual image by an occupant of the moving body and that includes content related to the object. The estimation unit compares the position information with estimated position information indicating the position of the object estimated based on the movement information of the moving body, to estimate a delay period including a delay from when the detection system detects the object to when the first acquisition unit acquires the position information. The correction unit corrects a shift in the display position of the content in the display image during the delay period due to movement of the moving body.

[0014] This has the advantage that, since the delay period is updated by the estimation unit, it is possible to improve the accuracy of correcting the deviation of the display position of the content related to the object in the display image compared to when the delay period is set to a fixed value. Also, since the delay period is updated by the estimation unit, it is not necessary to set the delay period in advance for each configuration (design) of the display system, which makes it easier to design the display system.

[0015] In a drawing system according to another aspect of the present disclosure, the estimation unit estimates the delay period by further referring to a first timestamp including the time measured by the detection system and a second timestamp including the time measured by the drawing system.

[0016] This has the advantage that when the detection system and the drawing system are started, the estimation unit only needs to calculate the time difference between the first timestamp and the second timestamp once, eliminating the need to sequentially estimate the delay period while the display system is started, thereby reducing the processing load on the estimation unit.

[0017] In a drawing system according to another aspect of the present disclosure, the estimation unit estimates the delay period after performing a disturbance removal process to remove a disturbance that affects the delay period.

[0018] This has the advantage that the estimation unit estimates the delay period after the disturbance has been removed, making it easier to improve the accuracy of estimating the delay period.

[0019] In a drawing system according to another aspect of the present disclosure, the disturbance removal process is a process that uses, as the delay period, an average value of calculated delay periods calculated in the process of estimating the delay period.

[0020] This has the advantage that it is easier to remove disturbances in the delay period compared to when the calculated delay period is used as the delay period every time it is calculated.

[0021] In a drawing system according to another aspect of the present disclosure, the disturbance removal process is a process that excludes the estimated result of the delay period during a period in which the interval between first timestamps indicating the time at which the detection system detected the object falls outside a predetermined range.

[0022] This has the advantage that it is easy to remove disturbances in the delay period by excluding the estimated results of the delay period during the period when the detection system is assumed to be unstable.

[0023] In a drawing system according to another aspect of the present disclosure, the disturbance removal process is a process of excluding the estimated result of the delay period during a period in which the amount of displacement of the orientation of the moving body is smaller than a predetermined value.

[0024] This has the advantage that, since the accuracy of estimating the delay period is likely to decrease during periods when the displacement of the moving body's orientation changes very little, such as when the moving body is moving on a straight road, excluding the delay period estimation results during such periods makes it easier to remove disturbances in the delay period.

[0025] In a drawing system according to another aspect of the present disclosure, the disturbance removal process is a process of excluding the estimated result of the delay period during a period in which the magnitude of vibration of the moving body is greater than a predetermined magnitude.

[0026] This has the advantage that since the accuracy of estimating the delay period tends to decrease during periods when the vibration of the moving body is relatively large, excluding the estimation results of the delay period during such periods makes it easier to remove disturbances in the delay period.

[0027] In a drawing system according to another aspect of the present disclosure, the correction unit corrects a deviation in the display position of the content in the display image after the display image is drawn by the drawing unit.

[0028] This has the advantage that it is possible to correct the deviation of the display position of the content in the display image while taking into consideration the delay that may occur in the rendering unit, making it easier to improve the accuracy of the correction.

[0029] In a drawing system according to another aspect of the present disclosure, the data transmitted from the drawing unit to the correction unit includes information detected by the detection system.

[0030] This has the advantage that it is easy to synchronize, for example, the position information of an object detected by the detection system with the display image drawn by the drawing unit.

[0031] In a drawing system according to another aspect of the present disclosure, the correction unit corrects a misalignment of the display position of the content in the display image by shifting the content drawn by the drawing unit.

[0032] This has the advantage that the processing load on the correction unit can be reduced and delays caused by processing by the correction unit can be reduced compared to correcting the shift in the display position of the content by redrawing the content.

[0033] In a drawing system according to another aspect of the present disclosure, the delay period includes a first delay period that is a delay from when the detection system detects the object to when the first acquisition unit acquires the position information, and a second delay period that is a delay after the first delay period until the drawing unit draws the display image. The correction unit includes a first correction unit and a second correction unit. The first correction unit corrects a deviation in the display position of the content on the display image during the first delay period before the drawing unit draws the display image. The second correction unit corrects a deviation in the display position of the content on the display image during the second delay period after the drawing unit draws the display image.

[0034] This has the advantage that the deviation of the display position of the content in the display image can be corrected both before and after the display image is drawn by the drawing unit, making it easier to further improve the accuracy of the correction.

[0035] In a drawing system according to another aspect of the present disclosure, the data transmitted from the drawing unit to the second correction unit includes information detected by the detection system.

[0036] This has the advantage that it is easy to synchronize, for example, the position information of an object detected by the detection system with the display image drawn by the drawing unit.

[0037] In a drawing system according to another aspect of the present disclosure, the second correction unit corrects a misalignment of the display position of the content in the display image by shifting the content drawn by the drawing unit.

[0038] This has the advantage that the processing load on the correction unit can be reduced and delays caused by processing by the correction unit can be reduced compared to correcting the shift in the display position of the content by redrawing the content.

[0039] In a drawing system according to another aspect of the present disclosure, the correction unit determines whether to use the first correction unit or the second correction unit to correct the shift in the display position of the content in the display image based on the magnitude of the displacement of the moving body.

[0040] This has the advantage that, for example, when the displacement of the moving body is small and the impact on the shift in the display position of the content is small, the processing load on the correction unit can be reduced by performing correction using only one of the first correction unit and the second correction unit.

[0041] In a drawing system according to another aspect of the present disclosure, the correction unit determines whether to use the first correction unit or the second correction unit to correct the shift in the display position of the content in the display image based on the magnitude of the change in the displacement of the moving body per unit time.

[0042] This has the advantage that, for example, when the change in the displacement of the moving body per unit time is small and the impact on the shift in the display position of the content is small, the processing load on the correction unit can be reduced by performing correction using only one of the first correction unit and the second correction unit.

[0043] In a drawing system according to another aspect of the present disclosure, the detection system includes a plurality of detection systems, and the estimation unit estimates the delay period for each of the plurality of detection systems.

[0044] This has the advantage that it is easier to improve the accuracy of correcting the deviation of the display position of the content in the displayed image, compared to when a single delay period is shared by multiple detection systems.

[0045] A display system according to one aspect of the present disclosure includes the rendering system and a projection unit, wherein the projection unit projects the display image rendered by the rendering system onto the display medium so that the display image is visually recognized as a virtual image by a passenger of the vehicle.

[0046] This has the advantage that the occupant can easily grasp the content related to the object by viewing the display image in which the deviation in the display position of the content has been corrected.

[0047] A display control system according to one aspect of the present disclosure includes a first acquisition unit, a second acquisition unit, an estimation unit, and a correction unit. The first acquisition unit acquires position information indicating the position of an object from a detection system that detects the object around a moving body. The second acquisition unit acquires movement information indicating a movement state of the moving body. The estimation unit compares the position information with estimated position information indicating the position of the object estimated based on the movement information of the moving body, thereby estimating a delay period including a delay from when the detection system detects the object to when the first acquisition unit acquires the position information. The correction unit corrects a shift in the display position of the content in a display image caused by movement of the moving body during the delay period. The display image is projected onto a display medium based on the position information and the movement information so that the display image can be viewed as a virtual image by a passenger of the moving body, and includes content related to the object.

[0048] This has the advantage that the occupant can easily grasp the content related to the object by viewing the display image in which the deviation in the display position of the content has been corrected.

[0049] A display control system according to one aspect of the present disclosure includes a display control unit that controls display of a display image. The display image is drawn based on position information indicating the position of an object acquired from a detection system that detects an object around a moving body and movement information indicating a movement state of the moving body, and is projected onto a display medium to be viewed as a virtual image by an occupant of the moving body, and includes content related to the object. The display control unit controls display of the display image so that a corrected display image is projected onto the display medium. The corrected display image is an image in which a shift in the display position of the content in the display image due to movement of the moving body during a delay period has been corrected. The delay period is a period including a delay between when the detection system detects the object and when the position information is acquired, the delay being estimated by comparing the position information with estimated position information indicating the position of the object estimated based on the movement information of the moving body.

[0050] This has the advantage that the occupant can easily grasp the content related to the object by viewing the display image in which the deviation in the display position of the content has been corrected.

[0051] A rendering method according to one aspect of the present disclosure includes a first acquisition step, a second acquisition step, a rendering step, an estimation step, and a correction step. In the first acquisition step, position information indicating the position of an object around a moving body is acquired from a detection system that detects the object. In the second acquisition step, movement information indicating a movement state of the moving body is acquired. In the rendering step, a display image including content related to the object is rendered based on the position information and the movement information, the display image being projected onto a display medium so that an occupant of the moving body can view the image as a virtual image. In the estimation step, a delay period including a delay from when the object is detected by the detection system to when the position information is acquired in the first acquisition step is estimated by comparing the position information with estimated position information indicating the position of the object estimated based on the movement information of the moving body. In the correction step, a shift in the display position of the content in the display image during the delay period due to movement of the moving body is corrected.

[0052] This has the advantage that, since the delay period is updated by the estimation step, it is possible to improve the accuracy of correcting the deviation of the display position of the content related to the object in the display image compared to when the delay period is set to a fixed value. Also, since the delay period is updated by the estimation step, it is not necessary to set the delay period in advance for each configuration (design) of the display system, which makes it easier to design the display system.

[0053] A program according to one aspect of the present disclosure causes one or more processors to execute the rendering method.

[0054] This has the advantage that, since the delay period is updated by the estimation step, it is possible to improve the accuracy of correcting the deviation of the display position of the content related to the object in the display image compared to when the delay period is set to a fixed value. Also, since the delay period is updated by the estimation step, it is not necessary to set the delay period in advance for each configuration (design) of the display system, which makes it easier to design the display system.

[0055] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, or a recording medium.

[0056] Hereinafter, the first and second embodiments will be described in detail with reference to the drawings. Note that each of the embodiments described below is a comprehensive or specific example. The numerical values, shapes, components, component arrangements, steps, step order, etc. shown in each of the following embodiments are examples and do not limit the present disclosure. Furthermore, among the components in each of the following embodiments, components not described in the independent claims are components that can be added arbitrarily.

[0057] Furthermore, each drawing is a schematic diagram and is not necessarily a precise illustration. In addition, the same components are denoted by the same reference numerals in each drawing.

[0058] (Embodiment 1) <Configuration> Fig. 1 is a block diagram showing an overview of a display system 200 having a drawing system 100 according to the first embodiment. Fig. 2 is a diagram showing an example of use of the display system 200 according to the first embodiment. The display system 200 according to the first embodiment is a head-up display (HUD) system that is mounted on a moving body 300 such as a vehicle 30, and projects an image representing information (display image 8) onto a predetermined area D1 of a display medium such as a windshield 32, thereby allowing a passenger having a viewpoint E1 to visually recognize a virtual image representing the information.

[0059] 2, the display system 200 is installed on the dashboard 31 of the vehicle 30. In the field of view of a passenger having a viewpoint E1, a virtual image is projected onto a virtual HUD display surface I1 and is visually recognized as being located in front of the vehicle 30. This display system allows information as a virtual image to be displayed in the field of view of the passenger, superimposed on the real scenery, etc.

[0060] In the first embodiment, the display system 200 applied to the vehicle 30 is described as an example of the moving body 300, but the moving body 300 is not limited to the vehicle 30 and may be, for example, a ship or an aircraft. In the first embodiment, the example is described in which the occupant of the moving body 300 is particularly the driver of the vehicle, but the occupant is not limited to the driver.

[0061] The display range of the virtual image displayed in the occupant's field of view by projecting an image in display system 200 (i.e., the maximum range in which the largest image can be displayed as a virtual image) is limited to a certain area, for example, by the arrangement or structure of display system 200. FIG. 3 is a diagram showing an example of a predetermined area D1 onto which an image displayed by display system 200 in embodiment 1 is projected. In other words, FIG. 3 is a diagram showing an example of the range of predetermined area D1 of windshield 32 (display medium) onto which an image displayed by display system 200 is projected, as seen from inside vehicle 30. As shown in FIG. 3, predetermined area D1 is an area corresponding to a certain angle of view (a certain viewing angle) in the field of view of an occupant (here, the driver) looking ahead of vehicle 30.

[0062] As shown in FIG. 1, the display system 200 includes a drawing system 100 and a projection unit 21.

[0063] The projection unit 21, under the control of the drawing system 100, projects an image onto the windshield 32 so that the driver of the vehicle 30 can view a virtual image (the image displayed on the HUD display surface I1). In other words, the projection unit 21 projects the display image 8 drawn by the drawing system 100 onto the windshield 32 (display medium) so that the occupants of the vehicle 30 (mobile body 300) can view it as a virtual image. The projection unit 21 has, for example, an LCD (Liquid Crystal Display), a flat mirror, and a concave mirror. The image drawn by the drawing system 100 is displayed on the LCD. The image displayed on the LCD is then reflected by the flat mirror, enlarged by the concave mirror, and then projected onto the windshield 32.

[0064] The drawing system 100 includes a computer including, for example, a memory and a processor (microprocessor), and the processor executes a control program stored in the memory to control each unit of the drawing system 100 and realize various functions. The memory may be a ROM (Read Only Memory) that stores programs and data in advance, a RAM (Random Access Memory) that is used to store data when a program is executed, or the like, and may include, for example, a non-volatile memory. The control program defines, for example, a display control process for displaying an image on a display medium to allow the driver of the vehicle 30 to view a virtual image.

[0065] The drawing system 100 has the function of acquiring various information from, for example, an object detection sensor 41 and a navigation device 42 as the detection system 400, and a vehicle information detection sensor 5, by executing a control program for display control processing by a processor, and controlling the display by the projection unit 21 based on the acquired various information.

[0066] The object detection sensor 41 is a sensor for advanced driver-assistance systems (ADAS), and is mounted inside or outside the vehicle 30. It sequentially (for example, every predetermined time such as 1 / 60 seconds) senses the view in front of the occupant (here, the driver) of the vehicle 30. In the first embodiment, the object detection sensor 41 includes an image sensor (camera) that detects light and the like in the view in front of it. The object detection sensor 41 may also include, for example, a radar that detects the reflection of electromagnetic waves. The object detection sensor 41 outputs information based on the results of the sensing to the drawing system 100.

[0067] The object detection sensor 41 recognizes, for example, an object 6 present in the foreground of the vehicle 30 around the vehicle (for example, within 100 m ahead), and outputs information such as the description and position of the object 6 as a recognition result. The object 6 is, for example, a moving object (for example, another vehicle or a pedestrian), a road sign, a white line on the road, a road marking, a curb, a guardrail, a traffic light, a utility pole, or a building. In the first embodiment, the object detection sensor 41 includes a camera, and therefore recognizes the object 6 by performing image processing such as pattern matching on an image of the foreground of the vehicle, which is the result of sensing. Also, for example, if the object detection sensor 41 is a radar, the object 6 is extracted and recognized from the information resulting from sensing using techniques such as clustering or machine learning. The object detection sensor 41 may identify the position of the object 6 as a relative position with respect to the vehicle 30, or may identify the absolute position using positioning information from a positioning system such as a GPS (Global Positioning System).

[0068] The navigation device 42 includes a positioning system such as a GPS receiver and has a vehicle navigation function based on positioning information and map data from the positioning system. The navigation device 42 may also include a storage device such as a memory or hard disk drive for acquiring and storing map data from an external device via communication, or a transceiver device. The navigation device 42 can measure the current position of the vehicle 30 using the positioning system and calculate the vehicle's traveling direction from the current position and previously measured positions of the vehicle 30. The navigation device 42 also recognizes objects 6 within 100 meters ahead of the vehicle 30 in the traveling direction based on the map data, and outputs information such as the description and position of the objects 6 as the recognition result to the rendering system 100. The objects 6 may be, for example, intersections.

[0069] The vehicle information detection sensor 5 is composed of various sensors (such as a speed sensor or a gyro sensor) that detect the state of the vehicle 30, and outputs information indicating the detected state to the drawing system 100. The state of the vehicle 30 may include, for example, the vehicle speed, the rotation speed (the rotation speed of the engine, etc.), the steering angle (the rotation angle of the steering wheel), the vibration of the vehicle 30, the gradient (pitch), the acceleration, or the yaw rate.

[0070] The drawing system 100 includes a first acquisition unit 11, a second acquisition unit 12, a drawing unit 13, an estimation unit 14, and a correction unit 15.

[0071] The first acquisition unit 11 acquires position information indicating the position of the object 6 from a detection system 400 that detects the object 6 around the vehicle 30 (mobile body 300). In the first embodiment, the position information indicates the relative position of the object 6 with respect to the vehicle 30. The first acquisition unit 11 is an entity that executes a first acquisition step ST1 in the drawing method. In the first embodiment, the first acquisition unit 11 acquires position information (e.g., position information of a person or the like in front of the vehicle 30) by acquiring a detection result from an object detection sensor 41 serving as the detection system 400. Also, in the first embodiment, the first acquisition unit 11 acquires position information (e.g., position information of an intersection or the like in front of the vehicle 30) by acquiring a recognition result from a navigation device 42 serving as the detection system 400. That is, in the first embodiment, the first acquisition unit 11 acquires position information from each of a plurality of (here, two) detection systems 400. The first acquisition unit 11 may acquire the position information from one detection system 400, or may acquire the position information from each of three or more detection systems 400.

[0072] The second acquisition unit 12 acquires movement information indicating the movement state of the vehicle 30 (moving body 300). The second acquisition unit 12 is an entity that executes the second acquisition step ST2 in the drawing method. In the first embodiment, the second acquisition unit 12 acquires the movement information by acquiring a detection result from the vehicle information detection sensor 5. Therefore, the movement information may include, for example, vehicle speed, rotation speed (rotation speed of the engine, etc.), steering angle (rotation angle of the steering wheel), gradient (pitch), acceleration, yaw rate, etc.

[0073] The drawing unit 13 draws a display image 8 that is projected onto a windshield 32 (display medium) to be visually recognized as a virtual image by an occupant (here, the driver) of the vehicle 30 (moving body 300) based on the position information acquired by the first acquisition unit 11 and the movement information acquired by the second acquisition unit 12, and that includes content 7 related to the object 6. The drawing unit 13 is an entity that executes a drawing step ST3 in the drawing method.

[0074] Specifically, the rendering unit 13 identifies the display position of the content 7 related to the object 6 at each predetermined display timing (for example, every 1 / 60 seconds) based on the position of the object 6 acquired by the first acquisition unit 11 and the position and traveling direction of the vehicle 30 (moving body 300) last acquired by the second acquisition unit 12. Then, the rendering unit 13 draws the display image 8 including the content 7 so that the content 7 is projected onto the windshield 32 (display medium) at the identified display position. Note that the display position of the content 7 is a position in a coordinate space for display and differs from a position in real space.

[0075] The content 7 is a display element formed by a mark such as a character string, a symbol, a graphic, or an image, and can be considered a component of a predetermined image displayed on the LCD of the projection unit 21 in one frame. The content 7 may include, for example, an image that emphasizes the object 6 in the display image 8. Specifically, if the object 6 is a person in front of the vehicle 30, the content 7 is a mark that emphasizes the person. The mark may include, for example, a circle surrounding the person's feet or a rectangular frame that entirely surrounds the person. If the object 6 is a road in front of the vehicle 30, the content 7 is an arrow indicating the direction in which the vehicle 30 should proceed according to the travel route set by the navigation device 42. The arrow may indicate, for example, going straight, turning right, or turning left. Note that the content 7 may be something other than a mark or an arrow as long as it is related to the object 6. For example, the content 7 may be a red warning line superimposed on a white lane dividing line when the vehicle 30 is about to enter an adjacent lane.

[0076] The correction unit 15 corrects a shift in the display position of the content 7 in the display image 8 during the delay period Td that occurs due to the movement of the vehicle 30 (moving body 300). The correction unit 15 is an entity that executes the correction step ST5 in the drawing method. Here, the delay period Td is a period that includes a delay from when the detection system 400 detects the object 6 to when the first acquisition unit 11 acquires the position information. In other words, the drawing system 100 cannot acquire the position of the content 7 without a time lag when the detection system 400 detects the position of the content 7, but acquires the position of the content 7 with a delay from that time. Therefore, when the display image 8 is drawn by referring to the position of the content 7 at the time when the first acquisition unit 11 acquires the position information, depending on the movement state of the vehicle 30 (moving body 300), a situation may occur in which the display position of the content 7 at that time is shifted compared to when the detection system 400 detected the position of the content 7. Therefore, the correction unit 15 corrects the deviation in the display position of the content 7, and the drawing unit 13 can draw the display image 8 so that the content 7 is displayed at the correct display position.

[0077] The process of correcting the deviation of the display position of the content 7 by the corrector 15 will be described below with reference to Fig. 4. Fig. 4 is an explanatory diagram of the correction process by the corrector 15 of the drawing system 100 according to the first embodiment.

[0078] In FIG. 4, a first area A1 illustrates a time series change of the vehicle 30 (moving body 300) relative to an object 6 (here, a person 61). A second area A2 illustrates an image of the area ahead of the vehicle 30 captured by the detection system 400 (here, a camera serving as the object detection sensor 41). A third area A3 illustrates a display image 8 drawn by the drawing unit 13 when the correction unit 15 does not perform the correction process. A fourth area A4 illustrates a display image 8 drawn by the drawing unit 13 when the correction unit 15 performs the correction process.

[0079] The first area A1 represents a situation in which the vehicle 30, traveling in the left lane, changes lanes from the left lane to the right lane while a person 61 is walking on a crosswalk ahead of the vehicle 30. In the first area A1, "t0" represents the start time, "Ts" represents the detection period of the detection system 400, and "Td" represents the delay period.

[0080] In the second area A2, images captured at times t0+Ts, t0+2Ts, t0+3Ts, and t0+4Ts are shown, starting from the bottom. The coordinates shown to the right of each image are XY coordinates that represent the position of person 61 (object 6) in the image, with the X-axis direction representing the horizontal direction in the image and the Y-axis direction representing the vertical direction in the image.

[0081] In the third area A3, from the bottom up, the display images 8 drawn by the drawing unit 13 at times t0+Ts+Td, t0+2Ts+Td, t0+3Ts+Td, and t0+4Ts+Td are shown, respectively, in a case where the correction unit 15 does not perform the correction process. In this way, the timing at which the display image 8 is drawn by the drawing unit 13 is delayed by the delay period Td from the timing at which the camera (detection system 400) captures (detects) the person 61 (target object 6).

[0082] Therefore, when the display position of the content 7 is identified with reference to the position information (XY coordinates) of the object 6 acquired from the detection system 400, there may be a misalignment between the display position of the object 6 and the display position of the content 7 in the display image 8. For example, at time t0+2Ts when the camera captures the image of the person 61, the vehicle 30 is located in the left lane. However, at time t0+2Ts+Td when the rendering unit 13 renders the display image 8, the vehicle 30 is in the process of moving from the left lane to the right lane, and the relative position of the person 61 with respect to the vehicle 30 has changed. Therefore, in the display image 8 rendered by the rendering unit 13 at time t0+2Ts+Td, the display position of the person 61 and the display position of the content 7 (here, a circle mark) are misaligned by the amount of displacement of the vehicle 30 in the X-axis direction during the delay period Td. A similar misalignment of the display positions occurs in the display image 8 rendered by the rendering unit 13 at time t0+3Ts+Td.

[0083] Therefore, the correction unit 15 calculates the amount of displacement of the vehicle 30 (moving object 300) during the delay period Td based on the movement information acquired by the second acquisition unit 12, and corrects the displacement of the display position of the content 7 in the display image 8 by shifting the display position of the content 7 by the calculated amount of displacement. Specifically, assuming that the position information of the object 6 acquired by the first acquisition unit 11 is coordinates Xn, Yn ("n" is a natural number), the correction unit 15 calculates relative displacement amounts ΔXsn, ΔYsn of the object 6 with respect to the vehicle 30 during the past delay period Td, which ends at the time when the position information was acquired, by referring to the movement information acquired by the second acquisition unit 12. "ΔXsn" represents the amount of displacement in the X-axis direction, and "ΔYsn" represents the amount of displacement in the Y-axis direction. The correction unit 15 then adds the calculated amounts of displacement ΔXsn, ΔYsn to the coordinates Xn, Yn of the object 6 to calculate coordinates X'n, Y'n that indicate the corrected display position of the content 7. The drawing unit 13 draws the display image 8 including the content 7, using the coordinates X'n, Y'n calculated by the correction unit 15 as the display position of the content 7.

[0084] A specific example of the correction process by the correction unit 15 will be described with reference to Fig. 4 and Fig. 5. Fig. 5 is a graph for explaining the correction process by the correction unit 15 of the drawing system 100 in the first embodiment. The following description focuses on the correction process for the position of the content 7 in the X-axis direction in the display image 8. Note that, although the description will be omitted, the correction process for the position of the content 7 in the Y-axis direction in the display image 8 is also similar.

[0085] 5, the vertical axis represents the position of the vehicle 30 in the X-axis direction, and the horizontal axis represents time. As shown in Fig. 5, for example, the position of the vehicle 30 in the X-axis direction at time t0+2Ts when the detection system 400 detects the object 6 is "Xc2," whereas the position of the vehicle 30 in the X-axis direction at time t0+2Ts+Td when the rendering unit 13 renders the display image 8 is "X'c2." In this case, the displacement amount ΔXs2 of the object 6 in the X-axis direction relative to the vehicle 30 during the delay period Td is calculated as the difference between "X'c2" and "Xc2."

[0086] The correction unit 15 calculates the coordinate X'2 in the X-axis direction of the object 6 at time t0+2Ts+Td by adding the calculated displacement amount ΔXs2 to the coordinate X2 in the X-axis direction of the object 6 at time t0+2Ts+Td, which is acquired by the first acquisition unit 11 at time t0+2Ts+Td. Here, the direction of the relative displacement of the object 6 with respect to the vehicle 30 is opposite to the direction of the displacement of the vehicle 30. Therefore, for example, when the displacement of the vehicle 30 is in a positive direction (here, rightward in the X-axis direction), the correction unit 15 adds a negative displacement amount to the coordinate of the object 6 so that the display position of the object 6 moves in a negative direction (here, leftward in the X-axis direction).

[0087] The rendering unit 13 renders the display image 8 by setting the coordinate X'2 of the object 6 in the X-axis direction calculated by the correction unit 15 as the coordinate X'2 of the content 7. As a result, as shown in the third area A3 of FIG. 4, in the display image 8 rendered by the rendering unit 13 at time t0+2Ts+Td, the display position of the person 61 and the display position of the content 7 (here, a circle mark) come to roughly coincide with each other. In other words, the correction unit 15 corrects the deviation in the display position of the content 7.

[0088] The estimation unit 14 estimates the delay period Td by comparing the position information acquired by the first acquisition unit 11 with estimated position information indicating the position of the object 6 estimated based on movement information of the vehicle 30 (moving body 300) acquired by the second acquisition unit 12. The estimation unit 14 is an entity that executes estimation step ST4 in the drawing method.

[0089] Here, the background behind the estimation of the delay period Td by the estimation unit 14 will be explained. As described above, if the displacement amount of the vehicle 30 during the delay period Td is calculated, it is possible for the correction unit 15 to correct the deviation of the display position of the content 7 in the display image 8. However, the delay period Td is not actually a uniquely determined value. For example, the delay period Td may vary depending on the type of vehicle 30, that is, the configuration (design) of the display system 200. Furthermore, the delay period Td may vary in real time depending on the state (e.g., processing load or communication status) of a processing unit of the display system 200, such as a central processing unit (CPU) or a graphics processing unit (GPU).

[0090] Therefore, if the correction unit 15 performs the above correction process with the delay period Td set to a fixed value, fluctuations in the delay period Td may prevent accurate calculation of the displacement of the vehicle 30, and as a result, it may not be possible to accurately correct the deviation of the display position of the content 7 in the display image 8. Therefore, the inventors of the present application have discovered that the accuracy with which the correction unit 15 corrects the deviation of the display position of the content 7 in the display image 8 can be improved by having the estimation unit 14 estimate the delay period Td and update the delay period Td.

[0091] The estimation process of estimating the delay period Td by the estimation unit 14 will be described below with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the operation of the drawing system 100 in the first embodiment. The tentative delay period Tdb in the following description is the delay period Td referred to by the correction unit 15 before the estimation unit 14 performs the estimation process, that is, the delay period Td before update.

[0092] First, at an arbitrary time t1, the first acquisition unit 11 acquires position information from the detection system 400 (S101). Then, at time t2 (= t1 + Ts), when a detection period Ts has elapsed since the time t1, the first acquisition unit 11 acquires position information from the detection system 400 (S102). The processes S101 and S102 correspond to the first acquisition step ST1 in the drawing method. Furthermore, the second acquisition unit 12 periodically acquires movement information of the vehicle 30 (moving object 300) from the vehicle information detection sensor 5 (S103). The process S103 corresponds to the second acquisition step ST2 in the drawing method.

[0093] In the following, the position information of the object 6 acquired by the first acquisition unit 11 at time t1 is referred to as coordinates Xn, Yn, and the position information of the object 6 acquired by the first acquisition unit 11 at time t2 is referred to as coordinates Xn+1, Yn+1. In the following description, it is assumed that the object 6 does not move between time t1 and time t2. In other words, the detection period Ts is a period in which the object 6 does not move, or moves very little, during this period.

[0094] The estimation unit 14 calculates the relative displacement amounts ΔXsn+1 and ΔYsn+1 of the object 6 with respect to the vehicle 30 over a predetermined period by referring to the history of movement information of the vehicle 30 (moving body 300) acquired by the second acquisition unit 12 (S104). The predetermined period starts at time t1', which is the provisional delay period Tdb before time t1, and ends at time t2', which is the detection period Ts after time t1'. These displacement amounts ΔXsn+1 and ΔYsn+1 may vary depending on how far back from time t1 the distance is taken, that is, depending on the value of the provisional delay period Tdb.

[0095] Next, the estimation unit 14 calculates estimated coordinates Xsn+1, Ysn+1, which are estimated value information of the object 6, by adding the calculated displacement amounts ΔXsn+1, ΔYsn+1 to the coordinates Xn, Yn of the object 6 acquired by the first acquisition unit 11 at time t1 (S105).Then, the estimation unit 14 compares the coordinates Xn+1, Yn+1 of the object 6 acquired by the first acquisition unit 11 at time t2 with the calculated estimated coordinates Xsn+1, Ysn+1 (S106).

[0096] The purpose of the above process S106 will now be described with reference to Fig. 7. Fig. 7 is a graph for explaining an example of the operation of the drawing system 100 in the first embodiment. In Fig. 7, the vertical axis represents the relative position of the object 6 in the X-axis direction with respect to the vehicle 30, and the horizontal axis represents time. Note that, although explanation will be omitted, the same applies to the relative position of the object 6 in the Y-axis direction with respect to the vehicle 30.

[0097] As shown in FIG. 7, the coordinate Xn of the object 6 acquired by the first acquisition unit 11 at time t1 is actually the coordinate of the object 6 detected by the detection system 400 at time t1-Td (see "P1" in FIG. 7). Furthermore, the coordinate Xn+1 of the object 6 acquired by the first acquisition unit 11 at time t2 (= t1+Ts) is actually the coordinate of the object 6 acquired by the detection system 400 at time t1-Td+Ts (see "P2" in FIG. 7). The coordinate Xn+1 of the object 6 at time t1-Td+Ts is calculated by adding the displacement ΔXsn+1 of the object 6 during the detection cycle Ts starting from time t1-Td to the coordinate Xn of the object 6 at time t1-Td. Therefore, if the tentative delay period Tdb matches the actual delay period Td, the coordinate Xn+1 of the object 6 will match the estimated coordinate Xsn+1. On the other hand, if the tentative delay period Tdb does not match the actual delay period Td, the displacement amount ΔXsn+1 will differ, and the coordinate Xn+1 of the target object 6 will not match the estimated coordinate Xsn+1.

[0098] If the result of the comparison shows that the difference between the coordinates Xn+1, Yn+1 of the object 6 and the estimated coordinates Xsn+1, Ysn+1 is within a predetermined range, in other words, these coordinates roughly match (S107: Yes), the estimation unit 14 sets the temporary delay period Tdb to the delay period Td, that is, updates the delay period Td to the temporary delay period Tdb (S108). Note that the term "update" here also includes the case where the delay period Td before the execution of the estimation process by the estimation unit 14 is maintained as it is.

[0099] On the other hand, if the difference between the coordinates Xn+1, Yn+1 of the object 6 and the estimated coordinates Xsn+1, Ysn+1 falls outside the predetermined range, in other words, if these coordinates do not match (S107: No), the estimation unit 14 updates the temporary delay period Tdb (S109) and executes the above-mentioned steps S104 to S106 again. In step S109, if the difference between the coordinates Xn+1, Yn+1 of the object 6 and the estimated coordinates Xsn+1, Ysn+1 is a positive value, the estimation unit 14 increases the temporary delay period Tdb by a predetermined value, and if the difference is a negative value, the estimation unit 14 decreases the temporary delay period Tdb by a predetermined value.

[0100] The estimation unit 14 repeats the above series of processes until the comparison result in process S106 indicates that the difference between the coordinates Xn+1, Yn+1 of the object 6 and the estimated coordinates Xsn+1, Ysn+1 falls within a predetermined range. The above processes S104 to S109 correspond to estimation step ST4 in the rendering method. This makes it possible to update the delay period Td.

[0101] The correction unit 15 corrects the deviation of the display position of the content 7 in the display image 8 using the delay period Td updated by the estimation unit 14 (S110). Process S110 corresponds to correction step ST5 in the drawing method. Then, the drawing unit 13 draws the display image 8 by referring to the display position of the content 7 corrected by the correction unit 15 (S111). Process S111 corresponds to drawing step ST3 in the drawing method. Processes S110 and S111 may be executed in parallel with processes S104 to S109, which correspond to estimation step ST4.

[0102] <Advantages> As described above, in the drawing system 100 according to the first embodiment, the estimation unit 14 estimates the delay period Td by comparing the position information acquired by the first acquisition unit 11 with estimated position information indicating the position of the object 6 estimated based on movement information of the vehicle 30 (moving body 300) acquired by the second acquisition unit 12. Therefore, in the first embodiment, the delay period Td is updated by the estimation unit 14, which has the advantage of improving the accuracy of correcting a shift in the display position of the content 7 related to the object 6 in the display image 8 compared to when the delay period Td is set to a fixed value. Furthermore, in the first embodiment, the delay period Td is updated by the estimation unit 14, which has the advantage of eliminating the need to set the delay period Td in advance for each configuration (design) of the display system 200, which makes it easier to design the display system 200.

[0103] (Embodiment 2) <Configuration> The drawing system 100 according to the second embodiment differs from the drawing system 100 according to the first embodiment in that the estimation unit 14 estimates the delay period Td by further referring to a first timestamp and a second timestamp. The first timestamp is information including the time measured by the detection system 400. For example, the first acquisition unit 11 acquires the position information of the object 6 from the detection system 400 together with the position information. The second timestamp is information including the time measured by the drawing system 100. For example, when the second acquisition unit 12 acquires the movement information of the moving object 300, the second timestamp is stored in memory together with the movement information.

[0104] Here, the first timestamp and the second timestamp are represented by the elapsed time since the detection system 400 was started up and the elapsed time since the drawing system 100 (display system 200) was started up, respectively. The first timestamp and the second timestamp may differ from each other due to, for example, differences in the start-up timing of the detection system 400 and the drawing system 100. Therefore, in the drawing system 100 of the second embodiment, the estimation unit 14 updates the delay period Td by calculating and updating the time difference between the first timestamp and the second timestamp, as will be described later.

[0105] <Operation> The process of estimating the delay period Td by the estimating unit 14 of the drawing system 100 according to the second embodiment will be described below with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the operation of the drawing system 100 according to the second embodiment.

[0106] First, at an arbitrary time t1, the first acquisition unit 11 acquires location information and a first timestamp from the detection system 400 (S201). Then, at time t2 (= t1 + Ts), when a detection period Ts has elapsed since the time t1, the first acquisition unit 11 acquires location information and a first timestamp from the detection system 400 (S202). The processes S201 and S202 correspond to a first acquisition step ST1 in the drawing method. Furthermore, the second acquisition unit 12 periodically acquires movement information of the vehicle 30 (moving object 300) from the vehicle information detection sensor 5 (S203). The process S203 corresponds to a second acquisition step ST2 in the drawing method.

[0107] In the following, the position information of the object 6 acquired by the first acquisition unit 11 at time t1 is designated as coordinates Xn, Yn, and the time indicated by the first timestamp acquired at time t1 is designated as time tIDn. Also, in the following, the position information of the object 6 acquired by the first acquisition unit 11 at time t2 is designated as coordinates Xn+1, Yn+1, and the time indicated by the first timestamp acquired at time t2 is designated as time tIDn+1. Note that in the following description, as in the first embodiment, it is assumed that the object 6 does not move between time t1 and time t2. In other words, the detection period Ts is a period in which the object 6 does not move, or moves very little, during this period.

[0108] The estimation unit 14 calculates the displacement amounts ΔXn+1, ΔYn+1 of the object 6 by calculating the difference between the coordinates Xn+1, Yn+1 acquired at time t2 and the coordinates Xn, Yn acquired at time t1 (S204). The estimation unit 14 also calculates the time difference ΔtIDn+1 by calculating the difference between the time tIDn+1 indicated by the first timestamp acquired at time t2 and the time tIDn indicated by the first timestamp acquired at time t1 (S205). ΔtIDn+1 corresponds to the detection period Ts measured by the detection system 400.

[0109] Next, the estimation unit 14 calculates estimated displacement amounts ΔXsn+1, ΔYsn+1, which are relative displacement amounts of the object 6 with respect to the vehicle 30 during a predetermined period, by referring to the history of movement information of the vehicle 30 (moving body 300) acquired by the second acquisition unit 12 (S206). The predetermined period is a period that starts at time t1', which is the tentative delay period Tdb before time t1, and ends at time t2', which is the detection period Ts after time t1'.

[0110] As already described, when the second acquirer 12 acquires the movement information of the moving object 300, the second timestamp is stored in memory together with the movement information. Here, if the time indicated by the second timestamp corresponding to time t2' is time tIDsn+1 and the time indicated by the second timestamp corresponding to time t1' is time tIDsn, the predetermined period is represented by the time difference ΔtIDsn+1, which is the difference between these times. The time tIDsn indicated by the second timestamp corresponding to time t1' can vary depending on the value of the tentative delay update period Tdb.

[0111] Then, the estimation unit 14 compares the displacement amounts ΔXn+1, ΔYn+1 of the object 6 based on the position information acquired by the detection system 400 with the estimated displacement amounts ΔXsn+1, ΔYsn+1 based on the movement information of the vehicle 30 (moving body 300) (S207). If the comparison shows that the difference between the displacement amounts ΔXn+1, ΔYn+1 of the object 6 and the estimated displacement amounts ΔXsn+1, ΔYsn+1 is within a predetermined range, in other words, these displacement amounts are approximately the same (S208: Yes), the estimation unit 14 sets the tentative delay period Tdb to the delay period Td, that is, updates the delay period Td to the tentative delay period Tdb (S209). In the second embodiment, the delay period Td is indirectly updated by updating the time difference ΔtIDd, ​​which is the difference between the time tIDsn indicated by the second timestamp corresponding to time t1′ and the time tIDn indicated by the first timestamp corresponding to time t1. Note that the case where the delay period Td before the execution of the estimation process by the estimation unit 14 is maintained as it is is also included in the term "updating" here.

[0112] On the other hand, if the difference between the displacement amounts ΔXn+1, ΔYn+1 of the object 6 and the estimated displacement amounts ΔXsn+1, ΔYsn+1 falls outside the predetermined range, in other words, if these displacement amounts do not match (S208: No), the estimation unit 14 updates the temporary delay period Tdb (S210) and executes the above steps S206 and S207 again. In step S210, if the difference in the displacement amounts is a positive value, the estimation unit 14 increases the temporary delay period Tdb by a predetermined value, and if it is a negative value, the estimation unit 14 decreases the temporary delay period Tdb by a predetermined value.

[0113] The estimation unit 14 repeats the above series of processes until the comparison result in process S207 indicates that the difference between the displacement amounts ΔXn+1, ΔYn+1 of the object 6 and the estimated displacement amounts ΔXsn+1, ΔYsn+1 falls within a predetermined range. The above processes S204 to S210 correspond to estimation step ST4 in the rendering method. This makes it possible to update the delay period Td.

[0114] Here, unlike in the first embodiment, the correction unit 15 of the drawing system 100 in the second embodiment corrects the deviation of the display position of the content 7 in the display image 8 using the time difference ΔtIDd. The correction process by the correction unit 15 of the drawing system 100 in the second embodiment will be described below with reference to FIG. 9. FIG. 9 is a flowchart showing an example of the correction process by the correction unit 15 of the drawing system 100 in the second embodiment. Note that the correction process by the correction unit 15 and the drawing process by the drawing unit 13 described below may be executed in parallel with the estimation process by the estimation unit 14 described above.

[0115] First, at an arbitrary time tn, the first acquisition unit 11 acquires position information and a first timestamp from the detection system 400 (S301). Process S301 corresponds to the first acquisition step ST1 in the drawing method. Hereinafter, the position information of the object 6 acquired by the first acquisition unit 11 at time tn is represented by coordinates Xn, Yn, and the time indicated by the first timestamp acquired at time tn is represented by time tIDn. Furthermore, the second acquisition unit 12 periodically acquires movement information of the vehicle 30 (moving object 300) from the vehicle information detection sensor 5 (S302). Process S302 corresponds to the second acquisition step ST2 in the drawing method.

[0116] Next, the correction unit 15 calculates corrected displacement amounts ΔXsn, ΔYsn, which are relative displacement amounts of the object 6 with respect to the vehicle 30 during a predetermined period, by referring to the history of movement information of the vehicle 30 (moving object 300) acquired by the second acquisition unit 12 and the time difference ΔtIDd estimated by the estimation unit 14 (S303). Here, the predetermined period is a period whose start point is the time point obtained by adding the time difference ΔtIDd to the above-mentioned time tIDn, and whose end point is time tn.

[0117] The correction unit 15 calculates coordinates X'n, Y'n indicating the corrected display position of the content 7 by adding the calculated corrected displacement amounts ΔXsn, ΔYsn to the coordinates Xn, Yn of the object 6 (S304). The processes S303 and S304 correspond to the correction step ST5 in the drawing method. Then, the drawing unit 13 draws the display image 8 including the content 7, using the coordinates X'n, Y'n calculated by the correction unit 15 as the display position of the content 7 (S305). The process S305 corresponds to the drawing step ST3 in the drawing method.

[0118] <Advantages> As described above, in the drawing system 100 according to the second embodiment, the estimation unit 14 estimates the delay period Td by further referring to a first timestamp including the time measured by the detection system 400 and a second timestamp including the time measured by the drawing system 100. Therefore, once the estimation unit 14 calculates the time difference ΔtIDd between the first timestamp and the second timestamp at the time of startup of the detection system 400 and the drawing system 100, it becomes unnecessary to sequentially estimate the delay period Td while the detection system 400 and the drawing system 100 are running, which has the advantage of reducing the processing load on the estimation unit 14.

[0119] (Variation) While the drawing system according to the present disclosure has been described above based on the above-mentioned embodiments, the present disclosure is not limited to these embodiments. As long as the modifications do not deviate from the spirit of the present disclosure, modifications that would occur to those skilled in the art may also be included within the scope of the present disclosure.

[0120] <First Modification> FIG. 10 is a block diagram showing an overview of a display system 200A having a rendering system 100A according to a first modification. In the first and second embodiments, the rendering unit 13 is located downstream of the correction unit 15, whereas in the first modification, the correction unit 15 is located downstream of the rendering unit 13 as shown in FIG. 10. That is, in the first modification, the correction unit 15 corrects a shift in the display position of the content 7 in the display image 8 after the rendering unit 13 renders the display image 8. Therefore, in the first modification, the delay period Td includes not only the delay from when the detection system 400 detects the object 6 until the first acquisition unit 11 acquires the position information, but also a delay that may occur in the rendering unit 13. The first modification has the advantage of easily improving the accuracy of the correction because the shift in the display position of the content 7 in the display image 8 can be corrected taking into account the delay that may occur in the rendering unit 13.

[0121] Furthermore, in the first modified example, when data communication is performed between the rendering unit 13 and the correction unit 15, the image data rendered by the rendering unit 13 may be transmitted to the correction unit 15 in a form in which the position information of the object 6, a timestamp, or the area of ​​the content 7 in the display image 8 is embedded. That is, in this aspect, the data transmitted from the rendering unit 13 to the correction unit 15 may include information detected by the detection system 400. This aspect has the advantage that, for example, it is easy to synchronize the position information of the object 6 detected by the detection system 400 with the display image 8 rendered by the rendering unit 13.

[0122] Furthermore, in the first modified example, when correcting a shift in the display position of content 7 in the display image 8, correction unit 15 may shift the already drawn content 7 to the corrected display position in the display image 8, rather than redrawing content 7. That is, correction unit 15 may correct a shift in the display position of content 7 in the display image 8 by shifting content 7 drawn by drawing unit 13. This aspect has the advantage that the processing load on correction unit 15 can be reduced and delays due to processing by correction unit 15 can be reduced, compared to when correcting a shift in the display position of content 7 by redrawing content 7.

[0123] <Second Modification> Fig. 11 is a block diagram showing an overview of a display system 200B having a drawing system 100B in the second modified example. In the second modified example, as shown in Fig. 11, the correction unit 15 has a first correction unit 151 located before the drawing unit 13 and a second correction unit 152 located after the drawing unit 13. That is, in the second modified example, before the drawing unit 13 draws the display image 8, the correction unit 15 corrects the deviation of the display position of the content 7 in the display image 8 by the first correction unit 151, and after the drawing unit 13 draws the display image 8, the correction unit 15 corrects the deviation of the display position of the content 7 in the display image 8 by the second correction unit 152.

[0124] Therefore, in the second modified example, the delay period Td includes a first delay period Td1, which is a delay from when the detection system 400 detects the object 6 until the first acquisition unit 11 acquires the position information, and a second delay period Td2, which is a delay after the first delay period Td1 until the rendering unit 13 renders the display image 8. The first correction unit 151 corrects a deviation in the display position of the content 7 in the display image 8 during the first delay period Td1 before the rendering unit 13 renders the display image 8. The second correction unit 152 corrects a deviation in the display position of the content 7 in the display image 8 during the second delay period Td2 after the rendering unit 13 renders the display image 8.

[0125] Here, the first delay period Td1 is a delay period estimated by the estimation unit 14, and the second delay period Td2 is a known, fixed value. If the second delay period Td2 is not known, the second delay period Td2 may be obtained by subtracting the first delay period Td1, which was also estimated by the estimation unit 14, from the delay period Td estimated by the estimation unit 14. If the second delay period Td2 is known, the first delay period Td1 may be obtained by subtracting the second delay period Td2 from the delay period Td estimated by the estimation unit 14.

[0126] 12 is a flowchart showing an example of the operation of the imaging system 100B in the second modification. First, the first correction unit 151 calculates a first displacement amount ΔX, which is a relative displacement amount of the object 6 with respect to the vehicle 30 during the first delay period Td1, based on the movement information of the vehicle 30 (moving body 300) acquired by the second acquisition unit 12. Td1 ,ΔY Td1 Then, the first correction unit 151 calculates the calculated first displacement amount ΔX Td1 ,ΔY Td1 The first corrector 151 then corrects the misalignment of the display position of the content 7 in the display image 8 input to the drawing unit 13 (S402). The drawing unit 13 draws the display image 8 using the coordinates of the content 7 corrected by the first corrector 151.

[0127] Next, the second correction unit 152 calculates a second displacement amount ΔX , which is a relative displacement amount of the object 6 with respect to the vehicle 30 during the second delay period Td2, based on the movement information of the vehicle 30 acquired by the second acquisition unit 12. Td2 ,ΔY Td2 Then, the second correction unit 152 calculates the calculated second displacement amount ΔX Td2 ,ΔY Td2 The second correction unit 152 corrects the deviation of the display position in the display image 8 of the content 7 output from the drawing unit 13 using the correction signal (S404). The projection unit 21 projects the display image 8 drawn by the drawing system 100B, that is, the display image 8 in which the display position of the content 7 has been corrected by the second correction unit 152, onto the windshield 32 (display medium).

[0128] Furthermore, in the second modified example, when data communication is performed between the rendering unit 13 and the second correction unit 152, the image data rendered by the rendering unit 13 may be transmitted to the second correction unit 152 in a form in which the position information of the object 6, a timestamp, or the area of ​​the content 7 in the display image 8 is embedded. That is, in this aspect, the data transmitted from the rendering unit 13 to the second correction unit 152 may include information detected by the detection system 400. This aspect has the advantage that, for example, it is easy to synchronize the position information of the object 6 detected by the detection system 400 with the display image 8 rendered by the rendering unit 13.

[0129] Furthermore, in the second modified example, when correcting a shift in the display position of content 7 in the display image 8, the second correction unit 152 may shift the already drawn content 7 to the corrected display position in the display image 8, rather than redrawing the content 7. That is, the second correction unit 152 may correct a shift in the display position of content 7 in the display image 8 by shifting the content 7 drawn by the drawing unit 13. This aspect has the advantage that the processing load on the correction unit 15 can be reduced and delays due to the processing of the correction unit 15 can be reduced, compared to when correcting a shift in the display position of content 7 by redrawing the content 7.

[0130] When the second correction unit 152 shifts the content 7 drawn by the drawing unit 13, correcting the shift in the display position of the content 7 using the first correction unit 151 before the drawing unit 13 has the following advantages.

[0131] FIG. 13 is an explanatory diagram of advantages of the drawing system 100B in the second modified example. (a) of FIG. 13 shows a display image 8 in a case where the deviation of the display position of the content 7 (here, a circular mark) relative to the object 6 (here, a person 61) is not corrected. (b) of FIG. 13 shows a display image 8 in a case where the deviation of the display position of the content 7 is corrected by the second corrector 152. (c) of FIG. 13 shows a display image 8 in a case where the size of the content 7 is corrected by the first corrector 151 and then the deviation of the display position of the content 7 is corrected by the second corrector 152. In the example shown in (a) to (c) of FIG. 13, the person 61 is located on the upper side of the display image 8, that is, far away from the vehicle 30. Therefore, in the display image 8, the person 61 is drawn at a relatively small size in consideration of the depth.

[0132] As shown in (b) of FIG. 13, when the shift in the display position of the content 7 is corrected only by the second correction unit 152, the content 7 has already been drawn by the drawing unit 13, so the size of the content 7 cannot be changed to match the size of the person 61, which causes a problem that it is difficult for the occupant to grasp the sense of distance of the virtual image. In contrast, as shown in (c) of FIG. 13, when the shift in the display position of the content 7 is further corrected by the first correction unit 151, the drawing unit 13 can draw the size of the content 7 relatively small to match the size of the person 61 based on the shift in the display position of the content 7 corrected by the first correction unit 151 (coordinates of the content 7), making it easier for the occupant to grasp the sense of distance of the virtual image. In other words, by correcting the shift in the display position of the content 7 by the first correction unit 151 located before the drawing unit 13, it is possible to change the shape or size of the content 7 depending on the display position of the content 7 in the display image 8.

[0133] FIG. 14 is an explanatory diagram of another advantage of the drawing system 100B in the second modified example. FIG. 14(a) shows a display image 8 in a case where a deviation in the display position of the content 7 (here, an arrow indicating a left turn) relative to the object 6 (here, an intersection 62 ahead of the vehicle 30) is not corrected. The lower display image 8 in FIG. 14(a) is an image in which the vehicle 30 is traveling in the left lane, and the upper display image 8 in FIG. 14(a) is an image in which the vehicle 30 is temporarily moving from the left lane to the right lane. FIG. 14(b) shows the display image 8 in a case where a deviation in the display position of the content 7 is corrected by the second correction unit 152. FIG. 14(c) shows the display image 8 in a case where the orientation of the content 7 is corrected by the first correction unit 151 and then the deviation in the display position of the content 7 is corrected by the second correction unit 152.

[0134] 14(a), in the upper display image 8, the deviation in the display position of the content 7 has not been corrected, and therefore the arrow does not correctly point to the intersection 62 where a left turn should be made. On the other hand, as shown in FIG. 14(b), when the deviation in the display position of the content 7 is corrected only by the second correction unit 152, the arrow correctly points to the intersection 62 where a left turn should be made, but the direction of the arrow does not follow the road, which causes a problem that it is difficult for the occupant to understand the instructions given by the content 7.

[0135] 14(c), when the first correction unit 151 further corrects the deviation in the display position of the content 7, the drawing unit 13 can draw the arrow so that its direction is aligned with the direction of the road based on the deviation in the display position of the content 7 (coordinates of the content 7) corrected by the first correction unit 151, making it easier for the occupant to understand the instructions of the content 7. In other words, by correcting the deviation in the display position of the content 7 by the first correction unit 151 located before the drawing unit 13, it is possible to change the direction of the content 7 according to the display position of the content 7 in the display image 8.

[0136] The first correction unit 151 may correct only the shape or size of the content 7 without correcting the display position of the content 7 in the display image 8. Because the correction of the shape or size of the content 7 is relatively less affected by delay, the first correction unit 151 may correct the shape or size of the content 7 using the delay period Td instead of the first delay time Td1. In this case, the first correction unit 151 calculates the amount of displacement of the object 6 relative to the vehicle 30 during the delay period Td, and corrects only the shape or size of the content 7 according to the calculated amount of displacement. Furthermore, in this case, the second correction unit 152 calculates the amount of displacement of the object 6 relative to the vehicle 30 during the delay period Td, and corrects the deviation of the display position of the content 7 in the display image 8 according to the calculated amount of displacement. The second correction unit 152 may use the amount of displacement calculated by the first correction unit 151. In this case, the second correction unit 152 does not need to calculate the amount of displacement, and the processing load can be reduced.

[0137] Incidentally, in the second modified example, the correction unit 15 may determine whether the first correction unit 151 or the second correction unit 152 should correct the shift in the display position of the content 7 in the display image 8, based on the magnitude of the amount of displacement of the vehicle 30 (moving body 300). For example, when the amount of displacement of the vehicle 30 is small and the effect on the shift in the display position of the content 7 is small, the processing load of the correction unit 15 can be reduced by performing the correction only with the first correction unit 151.

[0138] FIG. 15 is an explanatory diagram of another example of operation of the drawing system 100B in the second modified example. In FIG. 15, the vertical axis represents the displacement ΔY of the vehicle 30 in the Y-axis direction, and the horizontal axis represents time. As shown in FIG. 15, when the displacement ΔY falls within the range between the first threshold value Th1 and the second threshold value Th2, the correction unit 15 corrects the deviation of the display position of the content 7 in the display image 8 using only the first correction unit 151. On the other hand, when the displacement ΔY deviates from the above range, as shown in the circled area in FIG. 15, the correction unit 15 corrects the deviation of the display position of the content 7 in the display image 8 using both the first correction unit 151 and the second correction unit 152.

[0139] Furthermore, in the second modified example, the correction unit 15 may determine whether the first correction unit 151 or the second correction unit 152 should correct the shift in the display position of the content 7 on the display image 8, based on the magnitude of the change per unit time in the amount of displacement of the vehicle 30 (moving body 300) (in other words, the frequency of the fluctuations). For example, when the change per unit time in the amount of displacement of the vehicle 30 is small and the effect on the shift in the display position of the content 7 is small, the processing load of the correction unit 15 can be reduced by performing the correction only with the first correction unit 151.

[0140] FIG. 16 is an explanatory diagram of yet another example of operation of the drawing system 100B in the second modified example. In FIG. 16(a), the vertical axis represents the displacement ΔY of the vehicle 30 in the Y-axis direction, and the horizontal axis represents time. In FIG. 16(b), the vertical axis represents the derivative of the displacement ΔY, i.e., the change in the displacement ΔY per unit time, and the horizontal axis represents time. In FIG. 16(a), the frequency of fluctuation is relatively low in the region surrounded by a solid circle. On the other hand, in FIG. 16(a), the frequency of fluctuation is relatively high in the region surrounded by a dotted circle. The derivative of the displacement ΔY can be calculated, for example, by performing differentiation processing on the displacement ΔY or by performing high-pass filtering processing.

[0141] 16(b), when the derivative of the displacement amount ΔY is within the range between the third threshold value Th3 and the fourth threshold value Th4, the correction unit 15 corrects the deviation of the display position of the content 7 in the display image 8 only by the first correction unit 151. On the other hand, when the derivative of the displacement amount ΔY deviates from the above range as shown in the circled area in FIG. 16(b), the correction unit 15 corrects the deviation of the display position of the content 7 in the display image 8 by both the first correction unit 151 and the second correction unit 152.

[0142] Alternatively, the correction unit 15 may determine whether the first correction unit 151 or the second correction unit 152 is to correct the shift in the display position of the content 7 in the display image 8 for each of the displacement amount ΔX of the vehicle 30 (moving body 300) in the X-axis direction and the displacement amount ΔY of the vehicle 30 in the Y-axis direction. For example, the correction unit 15 may perform correction only with the first correction unit 151 for the displacement amount ΔX of the vehicle 30 in the X-axis direction, which is relatively less susceptible to the effects of delay, and may perform correction only with the second correction unit 152 for the displacement amount ΔY of the vehicle 30 in the Y-axis direction, which is relatively more susceptible to the effects of delay.

[0143] <Third Modification> Fig. 17 is a block diagram showing an overview of a display system 200C having a display control system 500 in the third modified example. As shown in Fig. 17, in the third modified example, a drawing system 100 is configured with a drawing unit 13 and a display control system 500 in the display system 200C. That is, the display control system 500 includes a first acquisition unit 11, a second acquisition unit 12, an estimation unit 14, and a correction unit 15, but does not include a drawing unit 13.

[0144] 17, the rendering unit 13 is located downstream of the correction unit 15, and the rendering system 100 includes the display control system 500, but this is not limiting. FIG. 18 is a block diagram showing an outline of another example of a display system 200C having the display control system 500 in the third modified example. As shown in FIG. 18, the rendering system 100C including the rendering unit 13 and the acquisition unit 131 may be separated from the display control system 500, and the correction unit 15 may be located downstream of the rendering unit 13. That is, in the example shown in FIG. 18, the correction unit 15 corrects a deviation in the display position of the content 7 in the display image 8 after the rendering unit 13 has rendered the display image 8.

[0145] The acquisition unit 131 has functions equivalent to those of the first acquisition unit 11 and the second acquisition unit 12. That is, the acquisition unit 131 acquires the position information from the detection system 400, and acquires the second information from the vehicle information detection sensor 5. Therefore, the drawing unit 13 draws the display image 8 based on the position information and movement information acquired by the acquisition unit 131.

[0146] Furthermore, in the third modified example, when data communication is performed between the rendering unit 13 and the correction unit 15, the image data rendered by the rendering unit 13 may be transmitted to the correction unit 15 in a form in which the position information of the object 6, a timestamp, or the area of ​​the content 7 in the display image 8 is embedded. That is, in this aspect, the data transmitted from the rendering unit 13 to the correction unit 15 may include information detected by the detection system 400. This aspect has the advantage that, for example, it is easy to synchronize the position information of the object 6 detected by the detection system 400 with the display image 8 rendered by the rendering unit 13.

[0147] <Fourth Modification> Fig. 19 is a block diagram showing an overview of a display system 200D having a display control system 500A in Modification 4. As shown in Fig. 19, in Modification 4, display system 200D has a display control system 500A including a display control unit 51 that controls the display of a display image 8, disposed between drawing unit 13 and projection unit 21.

[0148] The display image 8 is drawn based on position information indicating the position of the object 6 acquired from a detection system 400 that detects the object 6 around the vehicle 30 (moving body 300) and movement information indicating the movement state of the vehicle 30, and is projected onto the windshield 32 (display medium) so that the occupants of the vehicle 30 can view it as a virtual image, and includes content 7 related to the object 6. In other words, the display image 8 is an image output from the drawing system 100.

[0149] The display control unit 51 controls the display of the display image 8 so that the corrected display image 8 is projected onto the windshield 32 (display medium). The corrected display image 8 is an image in which a shift in the display position of the content on the display image 8 caused by the movement of the vehicle 30 during the delay period Td has been corrected. The delay period Td is a period including a delay from the time the detection system 1 detects the object 6 to the time the position information is acquired, which is estimated by comparing the position information with estimated position information indicating the position of the object 6 estimated based on movement information of the vehicle 30 (moving body 300).

[0150] <Other variations> In the first embodiment, for example, if the delay period Td is updated every time position information is acquired from the detection system 400, the delay period Td may not be accurately estimated due to the influence of disturbances. Disturbances may occur, for example, when the behavior of the vehicle 30 (moving body 300) is different from normal, or when the operation of the detection system 400 is unstable. Therefore, the estimation unit 14 may estimate the delay period Td after performing a disturbance removal process to remove disturbances that affect the delay period Td. This aspect has the advantage of making it easier to improve the accuracy of estimating the delay period Td, since the estimation unit 14 estimates the delay period Td after the disturbances have been removed. Examples of disturbance removal processes are listed below.

[0151] As an example, the disturbance removal process is a process in which the average value of the calculated delay period calculated in the process of estimating the delay period Td is used as the delay period Td. Specifically, as shown in FIG. 6, the estimation unit 14 calculates a temporary delay period Tdb that satisfies a condition and updates the calculated temporary delay period Tdb as the delay period Td. Here, the estimation unit 14 calculates the temporary delay period Tdb that satisfies the condition multiple times and uses the average value of the calculated temporary delay periods Tdb as the delay period Td. This mode has the advantage that disturbances in the delay period Td can be more easily removed compared to when the delay period Td is used each time a calculated delay period (a temporary delay period Tdb that satisfies a condition) is calculated. Note that the average value may be, for example, the average value of all previously calculated temporary delay periods Tdb, or the average value of a predetermined number of temporary delay periods Tdb going back from the time when the latest temporary delay period Tdb was calculated. Furthermore, the current average value may be maintained until the temporary delay period Tdb has been calculated a predetermined number of times, and may be updated when the temporary delay period Tdb has been calculated the predetermined number of times.

[0152] Additionally, as an example, the disturbance removal process is a process of excluding the estimation result of the delay period Td during a period in which the interval between first timestamps indicating the time when the detection system 400 detected the target object 6 (i.e., the detection period Ts) falls outside a predetermined range. In other words, if the detection period Ts is different from normal, it is assumed that the detection system 400 is unstable, and the delay period Td is not updated during such an unstable period. This aspect has the advantage that excluding the estimation result of the delay period Td during a period in which the detection system 400 is assumed to be unstable makes it easier to remove disturbances to the delay period Td.

[0153] Moreover, as an example, the disturbance removal process is a process of excluding the estimation result of the delay period Td during a period in which the amount of displacement of the orientation of the vehicle 30 (mobile body 300) is smaller than a predetermined value. That is, when the amount of displacement of the orientation of the vehicle 30 (e.g., the amount of displacement in the X-axis direction) is relatively small, for example, when the vehicle 30 is traveling on a straight road, it is difficult for the estimation unit 14 to accurately calculate the amount of displacement of the vehicle 30 during the predetermined period, and as a result, the accuracy of estimating the delay period Td may decrease. This aspect has the advantage that by excluding the estimation result of the delay period Td during such a period in which the amount of displacement of the orientation of the vehicle 30 is relatively small, disturbances of the delay period Td can be easily removed.

[0154] Moreover, as an example, the disturbance removal process is a process of excluding the estimation result of the delay period Td during a period in which the magnitude of vibration of the vehicle 30 (moving body 300) is greater than a predetermined magnitude. That is, when the vibration of the vehicle 30 is greater than a predetermined value due to, for example, poor road conditions or the vehicle 30 meandering, it is difficult for the estimator 14 to accurately calculate the displacement amount of the vehicle 30 during the predetermined period, and as a result, the accuracy of estimating the delay period Td may decrease. This aspect has the advantage of making it easier to remove disturbances from the delay period Td by excluding the estimation result of the delay period Td during such a period in which the vibration of the vehicle 30 is relatively large.

[0155] As in the first and second embodiments described above, there may be a plurality of detection systems 400 (in the first and second embodiments, the object detection sensor 41 and the navigation device 42 correspond to the detection system 400). In this case, the delay period Td may be a different value for each detection system 400. That is, the estimation unit 14 may estimate the delay period Td for each of the plurality of detection systems 400. This aspect has the advantage of making it easier to improve the accuracy of correcting a shift in the display position of the content 7 in the display image 8, compared to when one delay period Td is shared by the plurality of detection systems 400.

[0156] In the above-described first and second embodiments, when the detection system 400 detects a plurality of objects 6, the estimation unit 14 may estimate the delay period Td by focusing on any one of the plurality of objects 6. In this case, it is preferable that the object 6 to be focused on is, for example, an object with a small displacement amount per unit time, or an object closest to the center of the image captured by the camera serving as the object detection sensor 41, since this makes it easier to estimate the delay period Td.

[0157] The configuration of the projection unit 21 or the display method in the first and second embodiments is not limited to those described above, and any existing HUD-related technology may be applied and modified. For example, the projection unit 21 may use a laser projector or LCOS (Liquid Crystal on Silicon) instead of an LCD, and may use lenses instead of mirrors. The projection unit 21 may also use any number of mirrors. Furthermore, the projection unit 21 may be capable of adjusting the angle of a concave mirror, for example, by motor drive, depending on the height of the driver's viewpoint E1.

[0158] Furthermore, in the above-described embodiment, the projection unit 21 projects an image onto the windshield 32, but the image may be projected onto a combiner, which is a half mirror provided separately from the windshield 32. Furthermore, the projection unit 21 may be configured to display an image on a transmissive display provided inside or on the surface of the windshield 32.

[0159] Furthermore, the execution order of the steps of the display control process in the above-described drawing system 100 is not necessarily limited to the order described above. The execution order can be changed or some of the steps can be omitted without departing from the spirit of the invention. Furthermore, all or part of the steps of the above-described display control process can be implemented using hardware or software. Software processing is implemented by a processor included in a computer or the like executing a control program for the display control process stored in memory. The control program can also be recorded on a recording medium and distributed or circulated. For example, the distributed control program can be installed in a device having a processor and executed by the processor of the device, thereby causing the device to perform all or part of the display control process.

[0160] Furthermore, the computer in the above-described drawing system 100 may include, but is not limited to, an input device such as a touchpad, an output device such as a display or speaker, a storage device such as a hard disk drive or solid-state drive (SSD), a reading device that reads information from a recording medium such as a DVD-ROM (Digital Versatile Disk Read Only Memory) or a USB (Universal Serial Bus) memory, or a transmitting / receiving device that communicates via a network. For example, if the above-described control program is recorded on a recording medium such as a USB memory, the reading device reads the control program and stores it in memory or another storage device. Furthermore, the transmitting / receiving device may communicate with an external server device that stores the control program via a network, download the control program from the server device, and store it in memory or another storage device. The drawing system 100 may also be configured as an integrated circuit.

[0161] Furthermore, the scope of the present disclosure also includes forms realized by any combination of the above-described components and functions. [Industrial Applicability]

[0162] The present disclosure is applicable to a rendering system that renders a display image that is projected onto a display medium so that an occupant of a moving body can view it as a virtual image. [Explanation of symbols]

[0163] 11 First acquisition part 12 Second acquisition part 13 Drawing section 14 Estimation part 15 Correction unit 151 First Correction Section 152 Second Correction Section 21 Projection section 32 Windshield (display medium) 6. Objects 7 Content 8 Display image 100, 100A, 100B, 100C Drawing System 200, 200A, 200B, 200C, 200D Display Systems 300 Mobile 400 Detection System 500, 500A Display Control System ST1 First acquisition step ST2 Second acquisition step ST3 Drawing Step ST4 Estimation step ST5 Correction step Td delay period Td1 First delay period Td2 Second delay period

Claims

1. a first acquisition unit that acquires location information indicating a location of an object from a detection system that detects the object around the moving body; a second acquisition unit that acquires movement information indicating a movement state of the moving object; a rendering unit that renders a display image that is projected onto a display medium based on the position information and the movement information so that the display image is visually recognized as a virtual image by a passenger of the moving body, and that includes content related to the object; an estimation unit that estimates a delay period including a delay from when the detection system detects the object to when the first acquisition unit acquires the position information by comparing the position information with estimated position information that indicates the position of the object estimated based on movement information of the mobile body; and a correction unit that corrects a shift in the display position of the content in the display image caused by movement of the moving object during the delay period, the estimation unit estimates the delay period after performing a disturbance removal process to remove a disturbance that affects the delay period; The disturbance removal process is a process of excluding the estimation result of the delay period during a period in which an interval between first timestamps indicating the time at which the detection system detected the object falls outside a predetermined range. Drawing system.

2. the estimation unit estimates the delay period by further referring to a first timestamp including a time measured by the detection system and a second timestamp including a time measured by the drawing system. The drawing system according to claim 1 .

3. The disturbance removal process is a process of using an average value of the calculated delay period calculated in the process of estimating the delay period as the delay period. The drawing system according to claim 1 .

4. A first acquisition unit that acquires location information indicating the location of an object from a detection system that detects an object around a moving body; a second acquisition unit that acquires movement information indicating a movement state of the moving object; a rendering unit that renders a display image that is projected onto a display medium based on the position information and the movement information so that the display image is visually recognized as a virtual image by a passenger of the moving body, and that includes content related to the object; an estimation unit that estimates a delay period including a delay from when the detection system detects the object to when the first acquisition unit acquires the position information by comparing the position information with estimated position information that indicates the position of the object estimated based on movement information of the mobile body; and a correction unit that corrects a shift in the display position of the content in the display image caused by movement of the moving object during the delay period, the estimation unit estimates the delay period after performing a disturbance removal process to remove a disturbance that affects the delay period; the disturbance removal process is a process of excluding the estimation result of the delay period during a period in which the amount of displacement of the orientation of the moving object is smaller than a predetermined value. Drawing system.

5. A first acquisition unit that acquires location information indicating the location of an object from a detection system that detects an object around a moving body; a second acquisition unit that acquires movement information indicating a movement state of the moving object; a rendering unit that renders a display image that is projected onto a display medium based on the position information and the movement information so that the display image is visually recognized as a virtual image by a passenger of the moving body, and that includes content related to the object; an estimation unit that estimates a delay period including a delay from when the detection system detects the object to when the first acquisition unit acquires the position information by comparing the position information with estimated position information that indicates the position of the object estimated based on movement information of the mobile body; and a correction unit that corrects a shift in the display position of the content in the display image caused by movement of the moving object during the delay period, the estimation unit estimates the delay period after performing a disturbance removal process to remove a disturbance that affects the delay period; the disturbance removal process is a process of excluding the estimation result of the delay period during a period in which the magnitude of vibration of the moving body is greater than a predetermined magnitude. Drawing system.

6. the correction unit corrects a deviation of a display position of the content in the display image after the display image is drawn by the drawing unit; The drawing system according to any one of claims 1 to 5.

7. The data transmitted from the drawing unit to the correction unit includes information detected by the detection system. The drawing system according to claim 6 .

8. the correction unit corrects a deviation in the display position of the content in the display image by shifting the content drawn by the drawing unit.

8. The drawing system according to claim 6 or 7.

9. A first acquisition unit that acquires location information indicating the location of an object from a detection system that detects an object around a moving body; a second acquisition unit that acquires movement information indicating a movement state of the moving object; a rendering unit that renders a display image that is projected onto a display medium based on the position information and the movement information so that the display image is visually recognized as a virtual image by a passenger of the moving body, and that includes content related to the object; an estimation unit that estimates a delay period including a delay from when the detection system detects the object to when the first acquisition unit acquires the position information by comparing the position information with estimated position information that indicates the position of the object estimated based on movement information of the mobile body; and a correction unit that corrects a shift in the display position of the content in the display image caused by movement of the moving object during the delay period, the delay period includes a first delay period that is a delay from when the object is detected by the detection system to when the first acquisition unit acquires the position information, and a second delay period that is a delay after the first delay period until when the rendering unit renders the display image, The correction unit a first correction unit that corrects a deviation of a display position of the content in the display image during the first delay period before the display image is drawn by the drawing unit; a second correction unit that corrects a deviation of a display position of the content in the display image during the second delay period after the display image is drawn by the drawing unit, Drawing system.

10. The data transmitted from the drawing unit to the second correction unit includes information detected by the detection system. The drawing system according to claim 9 .

11. the second correction unit corrects a deviation in a display position of the content in the display image by shifting the content drawn by the drawing unit; 11. The drawing system according to claim 9 or 10.

12. the correction unit determines whether to use the first correction unit or the second correction unit to correct the deviation of the display position of the content in the display image, based on the magnitude of the displacement amount of the moving object. The drawing system according to any one of claims 9 to 11.

13. the correction unit determines whether to use the first correction unit or the second correction unit to correct the deviation of the display position of the content on the display image, based on the magnitude of the change per unit time in the amount of displacement of the moving object. The drawing system according to any one of claims 9 to 11.

14. The detection system is a plurality of the estimation unit estimates the delay period for each of the plurality of detection systems; The drawing system according to any one of claims 1 to 13.

15. A drawing system according to any one of claims 1 to 14; a projection unit that projects the display image drawn by the drawing system onto the display medium so that the display image is visually recognized as a virtual image by a passenger of the moving body, Display system.

16. a first acquisition unit that acquires location information indicating a location of an object from a detection system that detects the object around the moving body; a second acquisition unit that acquires movement information indicating a movement state of the moving object; an estimation unit that estimates a delay period including a delay from when the detection system detects the object to when the first acquisition unit acquires the position information by comparing the position information with estimated position information that indicates the position of the object estimated based on movement information of the mobile body; and a correction unit that corrects a shift in the display position of the content in the display image caused by movement of the moving object during the delay period, the display image is projected onto a display medium based on the position information and the movement information so as to be visually recognized as a virtual image by a passenger of the moving body, and includes content related to the object; the estimation unit estimates the delay period after performing a disturbance removal process to remove a disturbance that affects the delay period; The disturbance removal process is a process of excluding the estimation result of the delay period during a period in which an interval between first timestamps indicating the time at which the detection system detected the object falls outside a predetermined range. Display control system.

17. a display control unit that controls the display of a display image that is drawn based on position information indicating the position of an object obtained from a detection system that detects an object around the moving body and movement information indicating a movement state of the moving body, and that is projected onto a display medium so that an occupant of the moving body can view the image as a virtual image, and that includes content related to the object; the display control unit controls display of the display image so that the corrected display image is projected onto the display medium; the corrected display image is an image in which a shift in the display position of the content in the display image caused by movement of the moving object during a delay period has been corrected; the delay period is a period including a delay from the time when the detection system detects the object to the time when the position information is acquired, the period being estimated by comparing the position information with estimated position information indicating the position of the object estimated based on movement information of the mobile body; The delay period is estimated after performing a disturbance removal process to remove disturbances that affect the delay period; The disturbance removal process is a process of excluding the estimation result of the delay period during a period in which an interval between first timestamps indicating the time at which the detection system detected the object falls outside a predetermined range. Display control system.

18. A drawing method for a drawing system of an image to be projected onto a display medium of a moving body, comprising: a first acquisition step of acquiring position information indicating a position of an object from a detection system that detects an object around the moving body; a second acquisition step of acquiring movement information indicating a movement state of the moving object; a drawing step of drawing a display image, which is projected onto the display medium based on the position information and the movement information so that the display image is visually recognized as a virtual image by a passenger of the moving body, and which includes content related to the object; an estimation step of estimating a delay period including a delay from the time when the detection system detects the object to the time when the position information is acquired in the first acquisition step by comparing the position information with estimated position information indicating the position of the object estimated based on movement information of the mobile body; a correcting step of correcting a shift in the display position of the content on the display image caused by movement of the moving object during the delay period, In the estimation step, a disturbance removal process is performed to remove a disturbance that affects the delay period, and then the delay period is estimated; The disturbance removal process is a process of excluding the estimation result of the delay period during a period in which an interval between first timestamps indicating the time at which the detection system detected the object falls outside a predetermined range. How to draw.

19. one or more processors, Executing the drawing method according to claim 18, program.

Images