Display system and display method

Abstract

This display system (10) comprises: a position information acquisition unit (24) for acquiring position information indicating the position of a vanishing point calculated on the basis of two or more optical flows relating to each of a plurality of images captured by an imaging unit (2) that is mounted on a mobile body (1) and that repeatedly captures an image; a determining unit (26) for determining, on the basis of each of a plurality of items of position information acquired by the position information acquisition unit (24), whether each respective vanishing point from among a plurality of vanishing points satisfies a prescribed condition; and an estimation unit (28) for estimating the attitude of the mobile body (1) on the basis of one or more vanishing points, from among the plurality of vanishing points, determined by the determining unit (26) to satisfy the prescribed condition.

Description

【Technical Field】 【0001】 The present disclosure relates to a display system and a display method. 【Background Art】 【0002】 Conventionally, an attitude estimation device for estimating the attitude of a moving object has been known. For example, as an example of an attitude estimation device, Patent Document 1 discloses a calibration device that calculates camera installation parameters based on the positional relationship between the coordinates of two feature points in a camera image before movement and the coordinates of two feature points corresponding to the two feature points in the camera image after movement. 【Prior Art Documents】 【Patent Documents】 【0003】 【Patent Document 1】 International Publication No. 2019 / 106951 【Summary of the Invention】 【Problems to be Solved by the Invention】 【0004】 However, the calibration device of Patent Document 1 cannot accurately estimate the attitude of the moving object. 【0005】 Therefore, the present disclosure provides a display system and the like that can more accurately estimate the attitude of a moving object and can more appropriately display content. 【Means for Solving the Problems】 【0006】 A display system according to one aspect of the present disclosure includes: a position information acquisition unit that acquires position information indicating the position of a vanishing point calculated based on two or more optical flows for each of a plurality of images captured by an imaging unit mounted on a moving body and performing repeated imaging; a determination unit that determines whether each of the plurality of vanishing points satisfies a predetermined condition based on each of the plurality of position information acquired by the position information acquisition unit; and an estimation unit that estimates the posture of the moving body based on one or more vanishing points among the plurality of vanishing points that the determination unit has determined to satisfy the predetermined condition; an overlay position determination unit that determines the overlay position of content to be displayed superimposed on an object based on the posture of the moving body estimated by the posture estimation unit; and a display device that displays the content superimposed as a virtual image at the overlay position determined by the overlay position determination unit. 【0007】 A display method according to one aspect of the present disclosure includes: a position information acquisition step of acquiring position information indicating the position of a vanishing point calculated based on two or more optical flows for each of a plurality of images captured by an imaging unit mounted on a moving body and performing repeated imaging; a determination step of determining whether each of the plurality of vanishing points satisfies a predetermined condition based on each of the plurality of position information acquired by the position information acquisition step; an estimation step of estimating the posture of the moving body based on one or more vanishing points among the plurality of vanishing points that are determined to satisfy the predetermined condition by the determination step; a superposition position determination step of determining the superposition position of content to be displayed superimposed on an object based on the posture of the moving body estimated by the posture estimation step; and a display step of displaying the content superimposed as a virtual image at the superposition position determined by the superposition position determination step. 【0008】 These comprehensive or specific embodiments may be implemented as a system, method, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM, or as any combination of a system, method, integrated circuit, computer program, and recording medium. Furthermore, the recording medium may be a non-temporary recording medium. [Effects of the Invention] 【0009】 According to the display system and other features disclosed herein, the orientation of a moving object can be estimated with greater accuracy, and content can be displayed more appropriately. 【0010】 Further advantages and effects of one aspect of this disclosure will be made apparent from the specification and drawings. Such advantages and / or effects are provided by several embodiments and features described in the specification and drawings, but not all of them are necessarily provided in order to obtain one or more identical features. [Brief explanation of the drawing] 【0011】 [Figure 1] Figure 1 is a block diagram showing the functional configuration of the display system according to the embodiment. [Figure 2] Figure 2 is a flowchart showing an example of a display method using the display system shown in Figure 1. [Figure 3] Figure 3 is a flowchart showing an example of the operation included in step S4 of Figure 2. [Figure 4] Figure 4 is an explanatory diagram illustrating the calculation of optical flow and vanishing points using the display system shown in Figure 1. [Figure 5] Figure 5 is the first graph showing an example of multiple displacement amounts related to multiple images. [Figure 6] Figure 6 is a second graph showing an example of multiple displacement amounts related to multiple images. [Figure 7] Figure 7 is a graph showing an example of multiple standard deviations related to multiple images. [Figure 8]Figure 8 is a schematic diagram showing an example of the distance between a vanishing point and two or more optical flows. [Figure 9] Figure 9 is a graph showing an example of multiple coordinate values ​​related to multiple images. [Modes for carrying out the invention] 【0012】 A display system according to one aspect of the present disclosure includes: a position information acquisition unit that acquires position information indicating the position of a vanishing point calculated based on two or more optical flows for each of a plurality of images captured by an imaging unit mounted on a moving body and performing repeated imaging; a determination unit that determines whether each of the plurality of vanishing points satisfies a predetermined condition based on each of the plurality of position information acquired by the position information acquisition unit; and an estimation unit that estimates the posture of the moving body based on one or more vanishing points among the plurality of vanishing points that the determination unit has determined to satisfy the predetermined condition; an overlay position determination unit that determines the overlay position of content to be displayed superimposed on an object based on the posture of the moving body estimated by the posture estimation unit; and a display device that displays the content superimposed as a virtual image at the overlay position determined by the overlay position determination unit. 【0013】 According to this method, the orientation of a moving object can be estimated based on one or more vanishing points that are determined to satisfy predetermined conditions, and the use of vanishing points with poor calculation accuracy in estimating the object's orientation can be suppressed, thus enabling more accurate estimation of the object's orientation. Furthermore, because the object's orientation can be estimated more accurately, content can be displayed more appropriately. 【0014】 Furthermore, the predetermined conditions may include a first condition, wherein the first condition is defined as the vanishing point to be determined as the vanishing point to be determined, and the vanishing point of the image captured immediately before the image relating to the vanishing point to be determined among the plurality of images is defined as the vanishing point to be compared, and the amount of displacement of the vanishing point to be determined from the comparison vanishing point is within a predetermined range. 【0015】 According to this, when the displacement amount of the determination target vanishing point from the comparison vanishing point is not within a predetermined range, it is possible to suppress estimating the posture of the moving body using the determination target vanishing point. Therefore, the posture of the moving body can be estimated more accurately, and the content can be displayed more appropriately. 【0016】 Further, the predetermined condition may include a first condition. The first condition is that when the vanishing point that is the target of determination as to whether or not it satisfies the first condition is set as the determination target vanishing point, and the image related to the determination target vanishing point among the plurality of images is set as the determination target image, the variation degree of the two or more optical flows related to the determination target image with respect to the determination target vanishing point is within a predetermined range. 【0017】 According to this, when the variation degree of the two or more optical flows related to the determination target image with respect to the determination target vanishing point is not within a predetermined range, it is possible to suppress estimating the posture of the moving body using the determination target vanishing point. Therefore, the posture of the moving body can be estimated more accurately, and the content can be displayed more appropriately. 【0018】 Further, the predetermined condition may include a first condition. The first condition is that when the vanishing point that is the target of determination as to whether or not it satisfies the first condition is set as the determination target vanishing point, the determination target vanishing point is located within a predetermined range. 【0019】 According to this, when the determination target vanishing point is not located within a predetermined range, it is possible to suppress estimating the posture of the moving body using the determination target vanishing point. Therefore, the posture of the moving body can be estimated more accurately, and the content can be displayed more appropriately. 【0020】 Furthermore, the predetermined conditions may include a second condition that the vanishing point is not related to an image captured immediately after the image related to the vanishing point that was determined not to satisfy the first condition among the plurality of images, or that the vanishing point is not related to two or more images that are consecutive to the image captured immediately after the image related to the vanishing point that was determined not to satisfy the first condition among the plurality of images. 【0021】 According to this, it is possible to suppress the estimation of the pose of a moving object using the vanishing point of an image taken at a time close to the time when the image related to the vanishing point that was determined not to satisfy the first condition was taken. As a result, the pose of the moving object can be estimated with greater accuracy, and the content can be displayed more appropriately. 【0022】 Furthermore, the predetermined condition may also include a second condition that the vanishing point relating to two or more consecutive images among the plurality of images is not the vanishing point relating to the image immediately following the last image captured among two or more consecutive images in which the vanishing point relating to each of the plurality of images is determined not to satisfy the first condition, or that the vanishing point relating to two or more images is not the vanishing point relating to two or more images that are consecutive to the image immediately following the last image captured among two or more consecutive images in which the vanishing point relating to each of the plurality of images is determined not to satisfy the first condition. 【0023】 According to this, it is possible to suppress the estimation of the pose of a moving object using the vanishing points of images taken at close intervals to the timing when two or more consecutive images were taken, where the vanishing points related to each image are determined not to satisfy the first condition. As a result, the pose of the moving object can be estimated with greater accuracy, and content can be displayed more appropriately. 【0024】 Furthermore, the predetermined conditions may include a third condition, the third condition being that the amount of change in the velocity of the moving body during a predetermined period up to the time when the image relating to the vanishing point, which is the subject of the determination of whether or not the third condition is met among the plurality of images, is captured is within a predetermined range. 【0025】 According to this, it is possible to suppress the estimation of the moving object's attitude using the vanishing point in the image captured when the change in the moving object's velocity is outside a predetermined range, thereby enabling more accurate estimation of the moving object's attitude and more appropriate display of content. 【0026】 Furthermore, the estimation unit may estimate the attitude of the moving body based on two or more vanishing points from among the one or more vanishing points that the determination unit has determined to satisfy the predetermined conditions, and where the degree of variation among the two or more vanishing points is within a predetermined range. 【0027】 According to this method, the orientation of a moving object can be estimated based on two or more vanishing points whose degree of variation is within a predetermined range, thus enabling more accurate estimation of the object's orientation and more appropriate display of content. 【0028】 Furthermore, the system may further include an image acquisition unit for acquiring the plurality of images, an optical flow calculation unit for calculating the two or more optical flows for each of the plurality of images, and a vanishing point calculation unit for calculating the vanishing point for each of the plurality of images based on the two or more optical flows for each of the plurality of images. 【0029】 According to this, the display system can calculate two or more optical flows for each of the multiple images, and calculate the vanishing point for each of the multiple images. 【0030】 Furthermore, the optical flow calculation unit may, when it has selected two or more images from the plurality of images that are the subject of calculation for optical flow calculation as the calculation target images, and selected images taken before the calculation target images as the previous images, determine the previous images for each of the two or more optical flows related to the calculation target images, extract feature locations that appear in both the calculation target images and the determined previous images, calculate a straight line passing through a first feature point in the calculation target image that indicates the position of the feature location in the calculation target image and a second feature point in the calculation target image that indicates the position of the feature location in the determined previous image, and define the calculated straight line as the optical flow. 【0031】 According to this method, each of the two or more optical flows related to the image being calculated can be easily calculated. 【0032】 Furthermore, the optical flow calculation unit may determine, for at least one of the two or more optical flows related to the image to be calculated, that an image captured two or more times before the image to be calculated is the previous image. 【0033】 According to this, for at least one of the two or more optical flows related to the image being calculated, the distance between the first and second feature points can be made longer for more accurate calculation. As a result, vanishing points related to the image being calculated can be calculated with greater accuracy, the pose of moving objects can be estimated with greater accuracy, and content can be displayed more appropriately. 【0034】 Furthermore, the optical flow calculation unit may determine, for each of the two or more optical flows related to the image to be calculated, that an image captured two or more times before the image to be calculated is the previous image, with respect to 50% or more of the optical flows. 【0035】 According to this method, more than 50% of the optical flows in the two or more optical flows related to the image being calculated can be calculated with greater accuracy. As a result, vanishing points related to the image being calculated can be calculated with greater accuracy, the pose of the moving object can be estimated with greater accuracy, and the content can be displayed more appropriately. 【0036】 Furthermore, the optical flow calculation unit may determine the previous image such that, when the distance between the first feature point and the second feature point is defined as the feature point distance, the feature point distance for at least one of the two or more optical flows related to the image to be calculated is greater than or equal to a predetermined distance. 【0037】 According to this method, for at least one of two or more optical flows related to the image to be calculated, the distance between the first and second feature points can be set to a predetermined distance or greater to enable more accurate calculation. As a result, the vanishing point related to the image to be calculated can be calculated with greater accuracy, the pose of the moving object can be estimated with greater accuracy, and the content can be displayed more appropriately. 【0038】 Furthermore, a display method according to one aspect of the present disclosure includes: a position information acquisition step of acquiring position information indicating the position of a vanishing point calculated based on two or more optical flows for each of a plurality of images captured by an imaging unit mounted on a moving body and performing repeated imaging; a determination step of determining whether each of the plurality of vanishing points satisfies a predetermined condition based on each of the plurality of position information acquired by the position information acquisition step; an estimation step of estimating the posture of the moving body based on one or more vanishing points among the plurality of vanishing points that have been determined to satisfy the predetermined condition by the determination step; a superposition position determination step of determining the superposition position of content to be displayed superimposed on an object based on the posture of the moving body estimated by the posture estimation step; and a display step of displaying the content superimposed as a virtual image at the superposition position determined by the superposition position determination step. 【0039】 According to this, it will produce the same effects as the display system described above. 【0040】 The embodiments described below are all specific examples of this disclosure. The numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit this disclosure. Furthermore, any components in the following embodiments that are not described in an independent claim will be described as optional components. 【0041】 (Embodiment) Figure 1 is a block diagram showing the functional configuration of a display system 10 according to an embodiment. The functional configuration of the display system 10 will be described with reference to Figure 1. 【0042】 As shown in Figure 1, the display system 10 projects light toward the windshield of the mobile vehicle 1. At least a portion of the projected light is reflected by the windshield. This reflected light is directed toward the eyes of the user riding in the mobile vehicle 1. The user perceives this reflected light as a virtual image that appears as an image of an object on the opposite side of the windshield (outside the mobile vehicle 1), with real objects visible through the windshield as the background. This virtual image is displayed superimposed on the actual scenery visible within the user's field of view. Specifically, the virtual image is displayed superimposed on objects such as the road surface, buildings, surrounding vehicles, and pedestrians. For example, the mobile vehicle 1 is a vehicle. The display system 10 includes a posture estimation device 12, a superimposition position determination device 14, and a display device 16. 【0043】 The posture estimation device 12 estimates the posture of the moving object 1. The posture estimation device 12 includes an image acquisition unit 18, an optical flow calculation unit 20, a vanishing point calculation unit 22, a position information acquisition unit 24, a determination unit 26, and an estimation unit 28. 【0044】 The image acquisition unit 18 acquires multiple images captured by the imaging unit 2, which is mounted on the mobile body 1 and performs repeated imaging. For example, the image acquisition unit 18 acquires multiple images by acquiring the image each time the imaging unit 2 captures an image. For example, the imaging unit 2 is mounted on the mobile body 1 so as to be able to image the area in front of the mobile body 1, and repeatedly images the area in front of the mobile body 1. For example, the imaging unit 2 repeatedly captures images at predetermined time intervals, and the multiple images show the area in front of the mobile body 1 at predetermined time intervals. For example, the imaging unit 2 is a camera. 【0045】 The optical flow calculation unit 20 calculates two or more optical flows for each of the multiple images. For example, the optical flow calculation unit 20 calculates two or more optical flows for each of the multiple images by calculating two or more optical flows for each image each time the image acquisition unit 18 acquires an image. 【0046】 Specifically, for example, the optical flow calculation unit 20 calculates two or more optical flows related to the first image when the image acquisition unit 18 acquires the first image, calculates two or more optical flows related to the second image when the image acquisition unit 18 acquires the second image captured after the first image, and calculates two or more optical flows related to the third image when the image acquisition unit 18 acquires the third image captured after the second image. 【0047】 For example, the optical flow calculation unit 20, when it has selected two or more images from a plurality of images to be used for optical flow calculation as the calculation target images, and images taken before the calculation target images as the previous images, determines the previous image for each of the two or more optical flows related to the calculation target images, extracts feature locations that appear in both the calculation target image and the determined previous image, calculates a straight line passing through a first feature point in the calculation target image that indicates the location of the feature location in the calculation target image and a second feature point in the calculation target image that indicates the location of the feature location in the determined previous image, and defines the calculated straight line as the optical flow. For example, the first feature point is a pixel in the calculation target image that indicates the location of the feature location in the calculation target image. Also, for example, the second feature point is a pixel in the calculation target image that indicates the location of the feature location in the previous image. 【0048】 Specifically, for example, when the image acquisition unit 18 acquires a first image, a second image, and a third image, and calculates the first optical flow, second optical flow, and third optical flow related to the third image, the optical flow calculation unit 20 determines the first or second image captured before the third image as the preceding image for the first optical flow related to the third image, the first or second image captured before the third image as the preceding image for the second optical flow related to the third image, and the first or second image captured before the third image as the preceding image for the third optical flow related to the third image. Note that the preceding image for the first optical flow, the preceding image for the second optical flow, and the preceding image for the third optical flow may be different from each other. 【0049】 For example, if the optical flow calculation unit 20 determines the second image as the preceding image for the first optical flow related to the third image, it extracts feature locations that appear in both the third and second images. For example, feature locations may be the corners of a building or the corners of a building's doors. 【0050】 For example, if the optical flow calculation unit 20 extracts a "building corner" that appears in both the third image and the second image as a feature location, it calculates a straight line passing through a first feature point in the third image that indicates the position of the "building corner" in the third image and a second feature point in the third image that indicates the position of the "building corner" in the second image, and sets the calculated straight line as the first optical flow related to the third image. For example, the first feature point is a pixel in the third image that indicates the position of the "building corner" in the third image. Also, for example, the second feature point is a pixel in the third image that indicates the position of the "building corner" in the second image. 【0051】 Furthermore, for example, the optical flow calculation unit 20 determines, as a previous image, an image captured two or more times before the image to be calculated for at least one of the two or more optical flows related to the image to be calculated. For example, among the two or more optical flows related to the image to be calculated, the optical flow calculation unit 20 determines, as a previous image, an image captured two or more times before the image to be calculated for an optical flow in which the distance between the first feature point and the second feature point (hereinafter sometimes referred to as the feature point distance) is shorter than a predetermined distance when the image immediately preceding the image to be calculated is used as the previous image. 【0052】 Specifically, for example, when the image acquisition unit 18 acquires a first image, a second image, and a third image, and calculates the first optical flow, second optical flow, and third optical flow related to the third image, the optical flow calculation unit 20 determines the first image, which was captured two or more images before the third image, as the preceding image for at least one of the first optical flow, the second optical flow, and the third optical flow. For example, with respect to the second optical flow, the optical flow calculation unit 20 determines the first image as the preceding image if the feature point distance becomes shorter than a predetermined distance when the second image is used as the preceding image. 【0053】 Furthermore, for example, the optical flow calculation unit 20 determines, as the previous image, for each of the two or more optical flows related to the image to be calculated, to be an image captured two or more times before the image to be calculated, which accounts for 50% or more of the optical flows. For example, when the optical flow calculation unit 20 calculates 20 optical flows related to the image to be calculated, it determines, as the previous image, for each of the 10 or more optical flows among the 20 optical flows, which accounts for two or more times before the image to be calculated, which accounts for two or more of the optical flows related to the image to be calculated, which have a feature point distance shorter than a predetermined distance when the image immediately preceding the image to be calculated is used as the previous image. By doing this, the vanishing point can be calculated with even greater accuracy. 【0054】 Specifically, for example, when the image acquisition unit 18 acquires a first image, a second image, and a third image, and calculates the first optical flow, second optical flow, and third optical flow related to the third image, the optical flow calculation unit 20 determines the first image as the preceding image for each of the first optical flow, the second optical flow, and the third optical flow. 【0055】 Furthermore, for example, the optical flow calculation unit 20 determines a preceding image such that the feature point distance is greater than or equal to a predetermined distance for at least one of the two or more optical flows related to the image to be calculated, when the distance between the first feature point and the second feature point is defined as the feature point distance. For example, the optical flow calculation unit 20 prioritizes determining a preceding image such that the feature point distance is greater than or equal to a predetermined distance for the optical flow with a shorter feature point distance among the two or more optical flows related to the image to be calculated. 【0056】 Specifically, for example, when the image acquisition unit 18 acquires a first image, a second image, and a third image, and calculates the first optical flow, second optical flow, and third optical flow related to the third image, the optical flow calculation unit 20 determines the first image as the preceding image for the first optical flow if the feature point distance for the first optical flow becomes shorter than a predetermined distance when the second image is used as the preceding image, and the feature point distance for the first optical flow becomes greater than or equal to a predetermined distance when the first image is used as the preceding image. 【0057】 The vanishing point calculation unit 22 calculates the vanishing point for each of the multiple images based on two or more optical flows related to each of the multiple images. For example, each time two or more optical flows are calculated by the optical flow calculation unit 20, the vanishing point calculation unit 22 calculates the vanishing point for each of the multiple images based on those two or more optical flows. 【0058】 Specifically, for example, when the image acquisition unit 18 acquires a first image, a second image, and a third image, the vanishing point calculation unit 22 calculates the vanishing point for the first image based on two or more optical flows related to the first image, calculates the vanishing point for the second image based on two or more optical flows related to the second image, and calculates the vanishing point for the third image based on two or more optical flows related to the third image. 【0059】 For example, the vanishing point calculation unit 22 calculates the intersection point as the vanishing point for any image in which two or more optical flows intersect. 【0060】 For example, the vanishing point calculation unit 22 calculates a vanishing point as described later for images in which two or more optical flow intersections occur. For example, the vanishing point is the point where the absolute value of the difference between the first and second calculated values ​​falls within a predetermined range. The first calculated value is obtained by subtracting the average of the two or more distances from the longest distance among the two or more distances between the calculation point and the two or more optical flow, and the second calculated value is obtained by subtracting the shortest distance among the two or more distances from the average value. For example, the predetermined range is between 0 [pix] and 75 [pix]. 【0061】 Specifically, for example, if a first optical flow, a second optical flow, and a third optical flow have been calculated for an image to which the vanishing point is to be calculated, the vanishing point calculation unit 22 provisionally determines a calculation point and calculates a first distance between the calculation point and the first optical flow, a second distance between the calculation point and the second optical flow, and a third distance between the calculation point and the third optical flow. For example, if the first distance is the longest and the second distance is the shortest among the first, second, and third distances, the vanishing point calculation unit 22 calculates a first calculated value by subtracting the average value of the first, second, and third distances from the first distance, and calculates a second calculated value by subtracting the second distance from the average value. For example, if the absolute value of the difference between the first calculated value and the second calculated value is within a predetermined range, the vanishing point calculation unit 22 sets the provisionally determined calculation point as the vanishing point. For example, if the absolute value of the difference between the first calculated value and the second calculated value is not within a predetermined range, the vanishing point calculation unit 22 will tentatively determine the calculation point again and perform the above-described process again. 【0062】 The position information acquisition unit 24 acquires position information indicating the location of a vanishing point calculated based on two or more optical flows for each of the multiple images captured by the imaging unit 2, which is mounted on the mobile body 1 and performs repeated imaging. In other words, the position information acquisition unit 24 acquires position information indicating the location of a vanishing point for each of the multiple images. For example, the position information acquisition unit 24 acquires multiple pieces of position information for multiple images by acquiring position information for the vanishing point each time a vanishing point is calculated. In this embodiment, the position information acquisition unit 24 acquires position information indicating the location of a vanishing point calculated by the vanishing point calculation unit 22. For example, the position information is a coordinate value indicating the location of the vanishing point in the image. Specifically, for example, the position information is an X coordinate value and a Y coordinate value indicating the location of the vanishing point in the image. For example, the position information acquisition unit 24 may acquire position information indicating the location of a vanishing point calculated outside the attitude estimation device 12. 【0063】 The determination unit 26 determines whether each of the multiple vanishing points satisfies predetermined conditions based on each of the multiple location information acquired by the location information acquisition unit 24. For example, each time location information is acquired by the location information acquisition unit 24, the determination unit 26 determines whether each of the vanishing points related to that location information satisfies predetermined conditions based on that location information. 【0064】 For example, the predetermined conditions include a displacement condition, which is an example of the first condition. The displacement condition is a condition in which, when the vanishing point to be judged as the vanishing point to be judged as the vanishing point to be judged as the vanishing point to be judged as the vanishing point to be judged as the vanishing point to be compared 【0065】 The determination unit 26 determines that the vanishing point to be determined satisfies the displacement condition if the amount of displacement of the vanishing point to be determined from the comparison vanishing point is within a predetermined range. On the other hand, the determination unit 26 determines that the vanishing point to be determined does not satisfy the displacement condition if the amount of displacement of the vanishing point to be determined from the comparison vanishing point is not within a predetermined range. 【0066】 Specifically, for example, the displacement condition includes a condition regarding the displacement of the vanishing point to be judged in the X-axis direction from the comparison vanishing point, and a condition regarding the displacement of the vanishing point to be judged in the Y-axis direction from the comparison vanishing point. 【0067】 For example, the determination unit 26 calculates the amount of displacement of the vanishing point in the X-axis direction from the comparison vanishing point by using position information indicating the position of the vanishing point to be determined and position information indicating the position of the comparison vanishing point, and subtracting the X-coordinate value from the comparison vanishing point from the X-coordinate value of the vanishing point to be determined. Also, for example, the determination unit 26 calculates the amount of displacement of the vanishing point in the Y-axis direction from the comparison vanishing point by using position information indicating the position of the vanishing point to be determined and position information indicating the position of the comparison vanishing point, and subtracting the Y-coordinate value from the comparison vanishing point from the Y-coordinate value of the vanishing point to be determined. 【0068】 For example, the determination unit 26 determines that the vanishing point to be determined satisfies the displacement condition if the displacement amount in the X-axis direction from the comparison vanishing point of the vanishing point to be determined is within a predetermined range and the displacement amount in the Y-axis direction from the comparison vanishing point of the vanishing point to be determined is within a predetermined range. On the other hand, the determination unit 26 determines that the vanishing point to be determined does not satisfy the displacement condition if at least one of the displacement amount in the X-axis direction from the comparison vanishing point of the vanishing point to be determined and the displacement amount in the Y-axis direction from the comparison vanishing point of the vanishing point to be determined is outside the predetermined range. 【0069】 Furthermore, for example, when the determination unit 26 wants to estimate the pitch angle of the moving body 1, it determines that the vanishing point to be determined satisfies the displacement condition if the amount of displacement in the Y-axis direction from the comparison vanishing point is within a predetermined range. 【0070】 Furthermore, for example, when the determination unit 26 wants to estimate the yaw angle of the moving body 1, it determines that the vanishing point to be determined satisfies the displacement condition if the amount of displacement in the X-axis direction from the comparison vanishing point to the vanishing point to be determined is within a predetermined range. 【0071】 Furthermore, for example, the predetermined conditions include a variation condition, which is an example of the first condition. The variation condition is defined as a vanishing point that is the target vanishing point for determination, and when an image relating to the target vanishing point is selected from among multiple images, the variation of two or more optical flows relating to the target vanishing point is within a predetermined range. 【0072】 For example, the degree of variability is represented by the standard deviation of two or more distances between the vanishing point being judged and the two or more optical flows in question. 【0073】 For example, the determination unit 26 determines that a vanishing point satisfies the variability condition if the standard deviations of two or more distances between the vanishing point and the two or more optical flows are within a predetermined range. On the other hand, the determination unit 26 determines that a vanishing point does not satisfy the variability condition if the standard deviations of two or more distances between the vanishing point and the two or more optical flows are not within a predetermined range. 【0074】 Furthermore, for example, the degree of variability can be represented by the variance of two or more distances between the vanishing point being judged and the two or more optical flows in question. 【0075】 For example, the determination unit 26 determines that a vanishing point satisfies the variability condition if the variance of two or more distances between the vanishing point and the two or more optical flows is within a predetermined range. On the other hand, the determination unit 26 determines that a vanishing point does not satisfy the variability condition if the variance of two or more distances between the vanishing point and the two or more optical flows is not within a predetermined range. 【0076】 Furthermore, for example, the degree of variation is represented by the absolute value of the difference. For example, if the vanishing point that is the subject of the determination of whether or not the variation condition is met is defined as the vanishing point to be determined, and the image related to the vanishing point to be determined from among multiple images is defined as the image to be determined, and two or more distances between two or more optical flows related to the vanishing point to be determined and the image to be determined are defined as two or more distances to be determined, and the value obtained by subtracting the average value of the two or more distances to be determined from the longest distance is defined as the first target value, and the value obtained by subtracting the shortest distance to be determined from the average value is defined as the second target value, then the absolute value of the difference is the absolute value of the difference between the first target value and the second target value. 【0077】 For example, the determination unit 26 determines that the vanishing point to be determined satisfies the variation condition if the absolute value of the difference between the first target value and the second target value is within a predetermined range. On the other hand, the determination unit 26 determines that the vanishing point to be determined does not satisfy the variation condition if the absolute value of the difference between the first target value and the second target value is not within a predetermined range. 【0078】 Specifically, for example, if a first optical flow, a second optical flow, and a third optical flow have been calculated for the image to be judged, the judgment unit 26 calculates a first judgment target distance between the judgment target vanishing point and the first optical flow, a second judgment target distance between the judgment target vanishing point and the second optical flow, and a third judgment target distance between the judgment target vanishing point and the third optical flow. For example, the first judgment target distance is the length of the perpendicular line drawn from the judgment target vanishing point to the first optical flow, the second judgment target distance is the length of the perpendicular line drawn from the judgment target vanishing point to the second optical flow, and the third judgment target distance is the length of the perpendicular line drawn from the judgment target vanishing point to the third optical flow. In other words, each of the two or more judgment target distances is the length of the perpendicular line drawn from the judgment target vanishing point to each of the two or more optical flows. 【0079】 For example, if the first target distance is the longest and the second target distance is the shortest among the first, second, and third target distances, the determination unit 26 sets the value obtained by subtracting the average value of the first, second, and third target distances from the first target distance as the first target value, and the value obtained by subtracting the second target distance from the average value as the second target value. 【0080】 For example, the determination unit 26 determines that the vanishing point to be determined satisfies the variation condition if the absolute value of the difference between the first target value and the second target value is within a predetermined range. On the other hand, the determination unit 26 determines that the vanishing point to be determined does not satisfy the variation condition if the absolute value of the difference between the first target value and the second target value is not within a predetermined range. 【0081】 Furthermore, for example, the predetermined conditions include a positional condition, which is an example of the first condition. The positional condition is that, when the vanishing point to be judged as the vanishing point to be judged is defined as the vanishing point to be judged, the vanishing point to be judged is located within a predetermined range. 【0082】 The determination unit 26 determines that the vanishing point to be determined satisfies the positional condition if it is located within a predetermined range. On the other hand, the determination unit 26 determines that the vanishing point to be determined does not satisfy the positional condition if it is not located within a predetermined range. 【0083】 Specifically, for example, the positional conditions include conditions relating to the position of the vanishing point to be judged in the X-axis direction and conditions relating to the position of the vanishing point to be judged in the Y-axis direction. 【0084】 For example, the determination unit 26 obtains the position of the vanishing point to be determined in the X-axis direction and the position of the vanishing point to be determined in the Y-axis direction from position information indicating the position of the vanishing point to be determined. 【0085】 For example, the determination unit 26 determines that a vanishing point to be determined satisfies the position condition if the position of the vanishing point in the X-axis direction is within a predetermined range and the position of the vanishing point in the Y-axis direction is within a predetermined range. On the other hand, the determination unit 26 determines that a vanishing point to be determined does not satisfy the position condition if at least one of the position of the vanishing point in the X-axis direction and the position of the vanishing point in the Y-axis direction is not within a predetermined range. 【0086】 Furthermore, for example, the predetermined conditions include a first continuity condition, which is an example of a second condition. The first continuity condition is that the vanishing point is not related to an image captured immediately after an image related to a vanishing point that is determined not to satisfy the first condition (in this embodiment, the displacement condition, the difference value condition, and the position condition) among a plurality of images, or that the vanishing point is not related to two or more images that are consecutive to an image captured immediately after an image related to a vanishing point that is determined not to satisfy the first condition among a plurality of images. 【0087】 For example, the determination unit 26 determines that the vanishing point subject to determination satisfies the first continuity condition if it is not a vanishing point in an image captured immediately after the image related to the vanishing point that was determined not to satisfy the first condition among a group of images. Also, for example, the determination unit 26 determines that the vanishing point subject to determination does not satisfy the first continuity condition if it is a vanishing point in an image captured immediately after the image related to the vanishing point that was determined not to satisfy the first condition among a group of images. 【0088】 Specifically, for example, if the first continuity condition is that the vanishing point is not related to an image captured immediately after the image in which the vanishing point was determined not to satisfy the first condition among multiple images, and there is a first image, a second image captured immediately after the first image, and a third image captured immediately after the second image, and the vanishing point related to the first image is determined not to satisfy the first condition, then the determination unit 26 determines that the vanishing point related to the second image does not satisfy the first continuity condition. Also, for example, if the first continuity condition is that the vanishing point is not related to an image captured immediately after the image in which the vanishing point was determined not to satisfy the first condition among multiple images, and there is a first image, a second image captured immediately after the first image, and a third image captured immediately after the second image, and the vanishing point related to the second image is determined to satisfy the first condition, then the determination unit 26 determines that the vanishing point related to the third image satisfies the first continuity condition. 【0089】 Furthermore, for example, if the vanishing point to be determined for which the first continuity condition is met is not a vanishing point in two or more consecutive images starting from an image captured immediately after the image related to the vanishing point that was determined not to meet the first condition among a plurality of images, the determination unit 26 determines that the vanishing point to be determined for met satisfies the first continuity condition. Furthermore, for example, if the vanishing point to be determined for which the first continuity condition is met is a vanishing point in two or more consecutive images starting from an image captured immediately after the image related to the vanishing point that was determined not to meet the first condition among a plurality of images, the determination unit 26 determines that the vanishing point does not meet the first continuity condition. 【0090】 Specifically, for example, if the first continuity condition is that the vanishing point is not related to two consecutive images, starting from the image captured immediately after the image in which the vanishing point was determined not to satisfy the first condition among multiple images, and there is a first image, a second image captured immediately after the first image, and a third image captured immediately after the second image, and the vanishing point related to the first image is determined not to satisfy the first condition, the determination unit 26 determines that the vanishing point related to the second image and the vanishing point related to the third image do not satisfy the first continuity condition. 【0091】 Furthermore, for example, the specified conditions include a second continuity condition, which is an example of the second condition. The second continuity condition is that the vanishing point of two or more consecutive images among a group of images, each of which is determined not to satisfy the first condition, is not the vanishing point of the image captured immediately after the last image captured, or that the vanishing point of two or more consecutive images among a group of images, each of which is determined not to satisfy the first condition, is not the vanishing point of two or more images that are consecutive to the image captured immediately after the last image captured. 【0092】 For example, if the vanishing point to be determined for which the second continuity condition is met is in two or more consecutive images among a group of images, and the vanishing point related to each of those images is not the vanishing point of the image captured immediately after the last image captured of the two or more consecutive images that were determined not to meet the first condition, then the determination unit 26 determines that the vanishing point to be determined does not meet the second continuity condition. Also, for example, if the vanishing point to be determined for which the second continuity condition is met is in two or more consecutive images among a group of images, and the vanishing point related to each of those images is the vanishing point of the image captured immediately after the last image captured of the two or more consecutive images that were determined not to meet the first condition, then the determination unit 26 determines that the vanishing point does not meet the second continuity condition. 【0093】 Specifically, for example, the second continuity condition is that two consecutive images among multiple images, and the vanishing point related to each of them is not the vanishing point related to the image immediately following the last image captured among the two consecutive images that were determined not to satisfy the first condition, and there is a first image, a second image captured immediately following the first image, a third image captured immediately following the second image, and a fourth image captured immediately following the third image, and if it is determined that the vanishing point related to the first image and the vanishing point related to the second image do not satisfy the first condition, the determination unit 26 determines that the vanishing point related to the third image does not satisfy the second continuity condition. Furthermore, for example, the second continuity condition is that two consecutive images among multiple images, and the vanishing point related to each of them is not the vanishing point related to the image immediately following the last image captured among the two consecutive images that were determined not to satisfy the first condition, and there are a first image, a second image captured immediately following the first image, a third image captured immediately following the second image, and a fourth image captured immediately following the third image, and if it is determined that the vanishing point related to the third image satisfies the first condition, the determination unit 26 determines that the vanishing point related to the fourth image satisfies the second continuity condition. 【0094】 Furthermore, for example, if the vanishing point to be determined for which the second continuity condition is met is located in two or more consecutive images among a plurality of images, and the vanishing points related to each of these images are not two or more vanishing points related to two or more consecutive images starting from the image immediately following the last image captured in two or more consecutive images that were determined not to meet the first condition, then the determination unit 26 determines that the vanishing point to be determined does not meet the second continuity condition. Furthermore, for example, if the vanishing point to be determined for which the second continuity condition is met is located in two or more consecutive images among a plurality of images, and the vanishing points related to each of these images are two or more vanishing points related to two or more consecutive images starting from the image immediately following the last image captured in two or more consecutive images that were determined not to meet the first condition, then the determination unit 26 determines that the vanishing point does not meet the second continuity condition. 【0095】 Specifically, for example, the second continuity condition is that the vanishing points of two consecutive images among multiple images are not the two vanishing points of the two consecutive images starting from the image immediately following the last image captured among the two consecutive images that were determined not to satisfy the first condition, and there are a first image, a second image captured immediately following the first image, a third image captured immediately following the second image, and a fourth image captured immediately following the third image, and if it is determined that the vanishing points of the first image and the vanishing points of the second image do not satisfy the first condition, the determination unit 26 determines that the vanishing points of the third image and the vanishing points of the fourth image do not satisfy the second continuity condition. 【0096】 Furthermore, for example, the predetermined conditions include a velocity condition, which is an example of a third condition. The velocity condition is a condition in which the amount of change in the velocity of the moving object 1 during a predetermined period up to the time when the image relating to the vanishing point, which is the subject of the determination of whether or not the velocity condition is met among multiple images, is captured is within a predetermined range. 【0097】 The determination unit 26 determines that the vanishing point satisfies the speed condition if the amount of change in the speed of the moving object 1 during a predetermined period up to the time when the image relating to the vanishing point, which is the subject of the determination of whether or not the speed condition is met among multiple images, is captured, is within a predetermined range. On the other hand, the determination unit 26 determines that the vanishing point does not satisfy the speed condition if the amount of change in the speed of the moving object 1 during a predetermined period up to the time when the image relating to the vanishing point, which is the subject of the determination of whether or not the speed condition is met among multiple images, is not within a predetermined range. 【0098】 Specifically, for example, if the predetermined period is 1 second, the determination unit 26 calculates the change in the speed of the moving body 1 during the 1 second up to the time when the image relating to the vanishing point, which is the subject of the determination of whether or not the speed condition is met among multiple images, is captured, by subtracting the minimum speed from the maximum speed of the moving body 1 during the 1 second up to the time when the image is captured. Also, for example, if the predetermined period is 1 second, the determination unit 26 obtains the first speed of the moving body 1 at the time when the image relating to the vanishing point, which is the subject of the determination of whether or not the speed condition is met among multiple images, is captured, and the second speed of the moving body 1 at the time 1 second prior to the time when the image was captured, and calculates the change in the speed of the moving body 1 during the 1 second up to the time when the image was captured by subtracting the second speed from the first speed. For example, the determination unit 26 obtains the speed of the moving body 1 from a speed meter or the like mounted on the moving body 1. 【0099】 For example, if the determination unit 26 determines that the vanishing point related to the image taken during the predetermined period is not within a predetermined range, then the determination unit 26 may determine that the vanishing point related to the image taken during the predetermined period does not meet the predetermined conditions. 【0100】 The estimation unit 28 estimates the attitude of the moving body 1 based on one or more vanishing points from among a plurality of vanishing points that the determination unit 26 has determined to satisfy predetermined conditions. For example, these plurality of vanishing points are a plurality of vanishing points calculated from the time the moving body 1 started moving until the present. For example, each time a vanishing point is calculated, the estimation unit 28 estimates the attitude of the moving body 1 based on one or more vanishing points from among the plurality of vanishing points that the determination unit 26 has determined to satisfy predetermined conditions. 【0101】 For example, the estimation unit 28 calculates a first average value, which is the average of the X coordinate values ​​of the one or more vanishing points, and a second average value, which is the average of the Y coordinate values ​​of the one or more vanishing points, and calculates an average vanishing point, which is the point indicated by the first average value and the second average value. 【0102】 For example, each time the imaging unit 2 captures an image, the estimation unit 28 calculates the average vanishing point, calculates the distance between the center of the image and the average vanishing point, and calculates the attitude angle indicating the tilt of the moving object 1 from this distance and the focal length of the imaging unit 2. In this way, for example, the estimation unit 28 estimates the attitude of the moving object 1 at the time each of the multiple images was captured. 【0103】 Furthermore, for example, the estimation unit 28 estimates the posture of the moving body 1 based on two or more vanishing points from among the one or more vanishing points determined by the determination unit 26 to satisfy predetermined conditions, and where the degree of variation of those two or more vanishing points is within a predetermined range. For example, the degree of variation of those two or more vanishing points is expressed as the standard deviation of the positions of those two or more vanishing points. Alternatively, for example, the degree of variation of those two or more vanishing points is expressed as the variance of the positions of those two or more vanishing points. 【0104】 In other words, for example, if there are multiple vanishing points that the determination unit 26 has determined to satisfy a predetermined condition, the estimation unit 28 extracts two or more vanishing points from among the multiple vanishing points whose degree of variation falls within a predetermined range, and estimates the posture of the moving body 1 based on the two or more extracted vanishing points. 【0105】 Specifically, for example, if the determination unit 26 determines that the ten vanishing points satisfy predetermined conditions, the estimation unit 28 calculates the standard deviation of the positions of the ten vanishing points. For example, if the standard deviation is within a predetermined range, the estimation unit 28 calculates the average vanishing point of the moving body 1 based on the ten vanishing points and estimates the posture of the moving body 1 based on the average vanishing point. On the other hand, for example, if the standard deviation is not within a predetermined range, the estimation unit 28 calculates the standard deviation of the nine vanishing points, excluding the vanishing point furthest from the average position of the ten vanishing points, and determines whether the standard deviation is within a predetermined range. In this way, for example, if the standard deviation is not within a predetermined range, the estimation unit 28 reduces the number of vanishing points used to calculate the standard deviation and calculates the standard deviation again, and determines whether the newly calculated standard deviation is within a predetermined range. In this way, for example, the estimation unit 28 extracts two or more vanishing points from among the one or more vanishing points that the determination unit 26 has determined to satisfy a predetermined condition, and the degree of variation among the two or more vanishing points is within a predetermined range, calculates an average vanishing point based on the extracted two or more vanishing points, and estimates the posture of the moving body 1 based on the average vanishing point. 【0106】 The superimposition position determination device 14 determines the superimposition position of the content to be superimposed and displayed on the object, based on the orientation of the moving object 1 estimated by the orientation estimation device 12. For example, the superimposition position determination device 14 acquires data related to the content (HUD (Head-Up Display) display data) from outside the display system 10 and selects the content to be superimposed and displayed from this data. 【0107】 For example, when the superimposed position determination device 14 displays content superimposed on an object, it determines the superimposed position of the content based on the attitude angle of the mobile body 1 when the image including the object was captured. Specifically, for example, when the superimposed position determination device 14 displays content superimposed on an object, if the mobile body 1 is tilted upward when the image including the object was captured, it shifts the display position of the content downward compared to the display position when the mobile body 1 is not tilted. Also, when the superimposed position determination device 14 displays content superimposed on an object, if the mobile body 1 is tilted downward when the image including the object was captured, it shifts the display position of the content upward compared to the display position when the mobile body 1 is not tilted. In this way, for example, the superimposed position determination device 14 determines the superimposed position of the content to be displayed superimposed on the object. 【0108】 The display device 16 displays the content by superimposing it as a virtual image at the superposition position determined by the superposition position determination device 14. For example, the display device 16 displays the content by superimposing it onto an object by projecting light that forms the content toward the windshield of the moving body 1, so that the content is superimposed as a virtual image at the superposition position determined by the superposition position determination device 14. 【0109】 The functional configuration of the display system 10 has been explained above. 【0110】 Figure 2 is a flowchart showing an example of a display method using the display system 10 in Figure 1. Figure 3 is a flowchart showing an example of an operation included in step S4 of Figure 2. An example of a display method will be explained with reference to Figures 2 and 3. 【0111】 As shown in Figure 2, first, the image acquisition unit 18 acquires the image captured by the imaging unit 2 (step S1). For example, as described above, the image acquisition unit 18 acquires the image each time the imaging unit 2 captures an image, and acquires multiple images. 【0112】 When the image acquisition unit 18 acquires an image, the optical flow calculation unit 20 calculates two or more optical flows related to the image (step S2). For example, the optical flow calculation unit 20 calculates two or more optical flows related to the image as described above. 【0113】 Figure 4 is an explanatory diagram illustrating the calculation of optical flow and vanishing point using the display system 10 in Figure 1. Figure 4(a) shows the case where optical flow and vanishing point are calculated using the target image for which optical flow is to be calculated and the previous image taken immediately before the target image. Figure 4(b) shows the case where optical flow and vanishing point are calculated using the target image for which optical flow is to be to be calculated and the previous image taken two images before the target image. 【0114】 For example, as shown in Figure 4(a), the optical flow calculation unit 20 extracts the corner of the door 30 and the corner of the building 32 as two feature locations that appear in both the image to be calculated and the previous image taken immediately before the image to be calculated. When the optical flow calculation unit 20 extracts the corner of the door 30, it calculates a straight line 38 that passes through a first feature point 34 in the image to be calculated, which indicates the position of the corner of the door 30 in the image to be calculated, and a second feature point 36 in the image to be calculated, which indicates the position of the corner of the door 30 in the previous image. Similarly, when the optical flow calculation unit 20 extracts the corner of the building 32, it calculates a straight line 44 that passes through a first feature point 40 in the image to be calculated, which indicates the position of the corner of the building 32 in the image to be calculated, and a second feature point 42 in the image to be calculated, which indicates the position of the corner of the building 32 in the previous image. The optical flow calculation unit 20 uses the straight line 38 and the straight line 44 as the two optical flows related to the image to be calculated. 【0115】 For example, if the calculation target image and the previous image captured immediately before the calculation target image are used, and the distance between the first feature point 34 and the second feature point 36 is not greater than or equal to a predetermined distance, and the distance between the first feature point 40 and the second feature point 42 is not greater than or equal to a predetermined distance, the optical flow calculation unit 20 may use the calculation target image and the previous image captured two images before the calculation target image. 【0116】 For example, as shown in Figure 4(b), the optical flow calculation unit 20 extracts the corner of the door 30 and the corner of the building 32 as two feature points that appear in both the image to be calculated and the previous image taken two images prior to the image to be calculated. When the optical flow calculation unit 20 extracts the corner of the door 30, it calculates a straight line 48 passing through a first feature point 34, which is a point in the image to be calculated and indicates the position of the corner of the door 30 in the image to be calculated, and a second feature point 46, which is a point in the image to be calculated and indicates the position of the corner of the door 30 in the previous image. Similarly, when the optical flow calculation unit 20 extracts the corner of the building 32, it calculates a straight line 52 passing through a first feature point 40, which is a point in the image to be calculated and indicates the position of the corner of the building 32 in the image to be calculated, and a second feature point 50, which is a point in the image to be calculated and indicates the position of the corner of the building 32 in the previous image. The optical flow calculation unit 20 uses the straight line 48 and the straight line 52 as the two optical flows related to the image to be calculated. 【0117】 Returning to Figure 2, once the optical flow calculation unit 20 calculates two or more optical flows, the vanishing point calculation unit 22 calculates a vanishing point based on those two or more optical flows (step S3). For example, the vanishing point calculation unit 22 calculates a vanishing point based on those two or more optical flows as described above. 【0118】 For example, as shown in Figure 4(a), if the line 38 and the line 44 are calculated as two optical flows, the vanishing point calculation unit 22 calculates the intersection point 54 of the line 38 and the line 44 as the vanishing point. 【0119】 Furthermore, as shown in Figure 4(b), for example, if the line 48 and the line 52 are calculated as two optical flows, the vanishing point calculation unit 22 calculates the intersection point 56 of the line 48 and the line 52 as the vanishing point. 【0120】 Returning to Figure 2, once the vanishing point calculation unit 22 calculates the vanishing point, the attitude estimation device 12 estimates the attitude of the moving body 1 (attitude estimation step) (step S4). 【0121】 As shown in Figure 3, first, the position information acquisition unit 24 acquires position information indicating the location of the vanishing point (step S11). For example, as described above, the position information acquisition unit 24 acquires position information related to the vanishing point each time the vanishing point calculation unit 22 calculates a vanishing point, thereby acquiring position information indicating the location of the vanishing point calculated based on two or more optical flows for each of the multiple images captured by the imaging unit 2 (position information acquisition step). 【0122】 When the location information acquisition unit 24 acquires location information, the determination unit 26 determines, based on the location information, whether the vanishing point related to the location information satisfies predetermined conditions (step S12). For example, each time the location information acquisition unit 24 acquires location information, the determination unit 26 determines, based on the location information, whether the vanishing point satisfies predetermined conditions, thereby determining, based on each of the multiple locations acquired in the location information acquisition step, whether each of the multiple vanishing points satisfies predetermined conditions (determination step). For example, the determination unit 26 determines whether the vanishing point satisfies predetermined conditions in the manner described above. 【0123】 Figure 5 is the first graph, which shows an example of multiple displacement amounts related to multiple images (multiple frames). 【0124】 As shown in Figure 5, the determination unit 26 calculates the displacement amount of the vanishing point to be determined from the comparison vanishing point for each of the multiple images, and determines whether each of the multiple displacement amounts falls within a predetermined range. Here, the predetermined range is between -50 [pix / frm] and 50 [pix / frm]. 【0125】 The determination unit 26 determines that vanishing points related to images (frames) where the displacement amount falls within a predetermined range among multiple displacement amounts satisfy the predetermined conditions (displacement amount conditions). On the other hand, the determination unit 26 determines that vanishing points related to images where the displacement amount falls outside the predetermined range among multiple displacement amounts (see the dashed rectangle in Figure 5) do not satisfy the predetermined conditions. 【0126】 Figure 6 is a second graph showing an example of multiple displacement amounts related to multiple images. 【0127】 As shown in Figure 6, for example, the determination unit 26 may determine that the predetermined conditions are not met not only for the vanishing point of an image relating to a displacement amount that is not within a predetermined range among a plurality of displacement amounts, but also for the vanishing point of an image relating to a displacement amount relating to three consecutive images (see the dashed rectangle in Figure 6) that were captured immediately after the image in question. 【0128】 Figure 7 is a graph showing an example of multiple standard deviations related to multiple images. These standard deviations represent the standard deviations of two or more distances between the vanishing point and two or more optical flows. Figure 8 is a schematic diagram showing an example of the distances between the vanishing point and two or more optical flows. 【0129】 As shown in Figure 7, the determination unit 26 calculates the standard deviation of the distance between the vanishing point and two or more optical flows for each of the multiple images, and determines whether each of the multiple standard deviations falls within a predetermined range. Here, the predetermined range is between 0 pixels and 75 pixels. 【0130】 As shown in Figure 8, for example, if three optical flows 58, 60, and 62 have been calculated for the target image, three distances d0, d1, and d2 are calculated between the vanishing point 64 and the three optical flows 58, 60, and 62 for that frame, and the standard deviation of the three distances d0, d1, and d2 between the vanishing point 64 and the three optical flows 58, 60, and 62 is calculated using these three distances. 【0131】 Returning to Figure 7, the determination unit 26 determines that vanishing points in images relating to standard deviations that fall within a predetermined range among multiple standard deviations satisfy the predetermined conditions (variability conditions). On the other hand, the determination unit 26 determines that vanishing points in images relating to standard deviations that do not fall within the predetermined range among multiple standard deviations (see the dashed rectangle in Figure 7) do not satisfy the predetermined conditions. 【0132】 Figure 9 is a graph showing an example of multiple coordinate values ​​related to multiple images. Here, the coordinate values ​​are the X coordinates of the vanishing points. 【0133】 As shown in Figure 9, the determination unit 26 determines whether each of the multiple X coordinate values ​​relating to the multiple images falls within a predetermined range. Here, the predetermined range is between -100 [pix / frm] and 100 [pix / frm]. 【0134】 The determination unit 26 also determines whether the Y coordinate values ​​of each of the multiple vanishing points relating to the multiple images are within a predetermined range. For example, the predetermined range is between -100 [pix / frm] and 100 [pix / frm]. 【0135】 The determination unit 26 determines that the vanishing points in an image relating to X coordinates that are within a predetermined range among multiple X coordinates and Y coordinates that are within a predetermined range among multiple Y coordinates satisfy the predetermined conditions (position conditions). On the other hand, the determination unit 26 determines that the vanishing points in an image relating to X coordinates that are not within the predetermined range among multiple X coordinates (see the dashed rectangle in Figure 9) do not satisfy the predetermined conditions. 【0136】 Returning to Figure 3, if the determination unit 26 determines that the vanishing point satisfies predetermined conditions (Yes in step S12), the attitude estimation device 12 stores position information related to that vanishing point (step S13). For example, each time the determination unit 26 determines that the vanishing point satisfies predetermined conditions, the attitude estimation device 12 stores position information related to one or more vanishing points that satisfy predetermined conditions among a plurality of vanishing points. 【0137】 If the determination unit 26 determines that the vanishing point does not meet the predetermined conditions (No in step S12), the attitude estimation device 12 does not store the position information related to the vanishing point. 【0138】 The estimation unit 28 calculates an average vanishing point located at the average position of one or more vanishing points based on one or more stored positional information (step S14). For example, the estimation unit 28 calculates the average vanishing point as described above. 【0139】 The estimation unit 28 calculates the average vanishing point and then uses the average vanishing point to calculate the attitude angle of the moving object 1 (step S15). For example, the estimation unit 28 calculates the attitude angle of the moving object 1 using the average vanishing point as described above. 【0140】 Thus, the estimation unit 28 estimates the attitude of the moving body 1 based on one or more vanishing points that are determined to satisfy predetermined conditions in the determination step (estimation step). 【0141】 Returning to Figure 2, the superimposition position determination device 14 determines the superimposition position of the content to be superimposed and displayed on the object based on the orientation of the moving body 1 estimated in the orientation estimation step (superimposition position determination step) (step S5). For example, the superimposition position determination device 14 determines the superimposition position of the content to be superimposed and displayed on the object in the manner described above. 【0142】 The display device 16 displays the content as a virtual image superimposed at the superimposed position determined by the superimposed position determination step (display step) (step S6). For example, the display device 16 displays the content as a virtual image superimposed at the superimposed position determined by the superimposed position determination step, as described above. 【0143】 The above explains one example of a display method. 【0144】 In the display system 10, for example, if there is a dynamic change in the orientation of the moving object 1, the orientation of the moving object 1 can be estimated without using the vanishing point of the image captured at the time of the change. This allows for more accurate calculation of the static orientation of the moving object 1 and more appropriate display of the content. 【0145】 The above explains the display system 10, etc. 【0146】 The display system 10 according to this embodiment includes: a position information acquisition unit 24 that acquires position information indicating the position of vanishing points calculated based on two or more optical flows for each of a plurality of images captured by an imaging unit 2 mounted on a mobile body 1 that repeatedly performs imaging; a determination unit 26 that determines whether each of the plurality of vanishing points satisfies predetermined conditions based on each of the plurality of position information acquired by the position information acquisition unit 24; and an estimation unit 28 that estimates the posture of the mobile body 1 based on one or more vanishing points that the determination unit 26 has determined to satisfy predetermined conditions; an overlay position determination unit 14 that determines the overlay position of content to be displayed superimposed on an object based on the posture of the mobile body 1 estimated by the posture estimation unit 12; and a display device 16 that displays the content superimposed as a virtual image at the overlay position determined by the overlay position determination unit 14. 【0147】 According to this method, the posture of the mobile object 1 can be estimated based on one or more vanishing points that are determined to satisfy predetermined conditions, and the use of vanishing points with poor calculation accuracy to estimate the posture of the mobile object 1 can be suppressed, thereby enabling more accurate estimation of the posture of the mobile object 1. Furthermore, because the posture of the mobile object 1 can be estimated more accurately, the content can be displayed more appropriately. 【0148】 Furthermore, the predetermined conditions include the first condition (displacement amount condition), which is the condition in which the vanishing point to be determined as the vanishing point to be determined, and the vanishing point of the image captured immediately before the image related to the vanishing point to be determined among multiple images is the vanishing point, and the displacement amount of the vanishing point to be determined from the comparison vanishing point is within a predetermined range. 【0149】 According to this, if the displacement of the vanishing point to be judged from the comparison vanishing point is not within a predetermined range, the estimation of the attitude of the moving object using the vanishing point to be judged can be suppressed, allowing for more accurate estimation of the attitude of the moving object 1 and more appropriate display of the content. 【0150】 Furthermore, the predetermined conditions include the first condition (variability condition), which is the condition that, when the vanishing point that is the subject of the determination to determine whether or not the first condition (variability condition) is met is designated as the vanishing point to be determined, and when the image relating to the vanishing point to be determined is designated as the image to be determined among multiple images, the degree of variation of two or more optical flows relating to the image to be determined with respect to the vanishing point to be determined is within a predetermined range. 【0151】 According to this, if the degree of variation of two or more optical flows related to the image to be judged with respect to the vanishing point to be judged is not within a predetermined range, the estimation of the pose of the moving object 1 using the vanishing point to be judged can be suppressed, thereby enabling more accurate estimation of the pose of the moving object 1 and more appropriate display of the content. 【0152】 Furthermore, the specified conditions include the first condition (position condition), which is the condition that when the vanishing point subject to determination as the vanishing point subject to determination is defined as the vanishing point subject to determination, the vanishing point subject to determination is located within a specified range. 【0153】 According to this, if the vanishing point to be determined is not located within a predetermined range, the estimation of the posture of the moving object 1 using the vanishing point to be determined can be suppressed, thereby enabling more accurate estimation of the posture of the moving object 1 and more appropriate display of content. 【0154】 Furthermore, the predetermined conditions include a second condition that the vanishing point is not related to an image captured immediately after an image related to a vanishing point that is determined to not satisfy the first condition (displacement condition, variation condition, and position condition) among multiple images, or that it is not related to two or more vanishing points in two or more consecutive images starting from an image captured immediately after an image related to a vanishing point that is determined to not satisfy the first condition among multiple images. 【0155】 According to this, it is possible to suppress the estimation of the pose of a moving object using the vanishing point of an image taken at a time close to the time when the image related to the vanishing point that was determined not to satisfy the first condition was taken. As a result, the pose of the moving object can be estimated with greater accuracy, and the content can be displayed more appropriately. 【0156】 Furthermore, the predetermined conditions include a second condition that the vanishing point is not the vanishing point of the image immediately following the last image captured in a set of two or more consecutive images from which the vanishing point of each of the multiple images is determined not to satisfy the first condition, or that the vanishing point is not the vanishing point of two or more images that are consecutive to the image immediately following the last image captured in a set of two or more consecutive images from which the vanishing point of each of the multiple images is determined not to satisfy the first condition. 【0157】 According to this, it is possible to suppress the estimation of the pose of the moving object 1 using the vanishing points of images taken at close timing to the timing when two or more consecutive images were taken in which it was determined that the vanishing points related to each of them do not satisfy the first condition. As a result, the pose of the moving object 1 can be estimated with greater accuracy, and the content can be displayed more appropriately. 【0158】 Furthermore, the specified conditions include a third condition (velocity condition), which is that the amount of change in the velocity of the moving object 1 during a predetermined period up to the time when the image relating to the vanishing point, which is the subject of the determination of whether or not the third condition (velocity condition) is met among multiple images, is captured, is within a predetermined range. 【0159】 According to this, it is possible to suppress the estimation of the attitude of the moving object 1 using the vanishing point in the image captured when the amount of change in the speed of the moving object 1 is outside a predetermined range, thereby enabling more accurate estimation of the attitude of the moving object 1 and more appropriate display of content. 【0160】 Furthermore, the estimation unit 28 estimates the posture of the moving body 1 based on two or more vanishing points from among the one or more vanishing points determined by the determination unit 26 to satisfy predetermined conditions, where the degree of variation among the two or more vanishing points is within a predetermined range. 【0161】 According to this, the orientation of the mobile object 1 can be estimated based on two or more vanishing points whose degree of variation is within a predetermined range, so the orientation of the mobile object 1 can be estimated with greater accuracy, and the content can be displayed more appropriately. 【0162】 The system further includes an image acquisition unit 18 that acquires multiple images, an optical flow calculation unit 20 that calculates two or more optical flows for each of the multiple images, and a vanishing point calculation unit 22 that calculates a vanishing point for each of the multiple images based on the two or more optical flows for each of the multiple images. 【0163】 According to this, the display system 10 can calculate two or more optical flows for each of the multiple images, and calculate vanishing points for each of the multiple images. 【0164】 Furthermore, the optical flow calculation unit 20, when it has selected two or more images from a plurality of images to be used for optical flow calculation as the calculation target images, and selected images taken before the calculation target images as the prior images, determines the prior images for each of the two or more optical flows related to the calculation target images, extracts feature locations that appear in both the calculation target image and the determined prior images, calculates a straight line passing through a first feature point in the calculation target image that indicates the location of a feature location in the calculation target image and a second feature point in the calculation target image that indicates the location of a feature location in the determined prior image, and defines the calculated straight line as the optical flow. 【0165】 According to this method, each of the two or more optical flows related to the image being calculated can be easily calculated. 【0166】 Furthermore, the optical flow calculation unit 20 determines, for at least one of the two or more optical flows related to the image to be calculated, an image captured two or more times before the image to be calculated as the previous image. 【0167】 According to this, for at least one of the two or more optical flows related to the image to be calculated, the distance between the first feature point and the second feature point can be made longer for more accurate calculation. As a result, the vanishing point related to the image to be calculated can be calculated with greater accuracy, the pose of the moving object 1 can be estimated with greater accuracy, and the content can be displayed more appropriately. 【0168】 Furthermore, the optical flow calculation unit 20 determines, for each of the two or more optical flows related to the image to be calculated, that 50% or more of them are images captured two or more times before the image to be calculated, as the preceding image. 【0169】 According to this method, more than 50% of the optical flows in the two or more optical flows related to the image to be calculated can be calculated with greater accuracy. As a result, vanishing points related to the image to be calculated can be calculated with greater accuracy, the pose of the moving object 1 can be estimated with greater accuracy, and the content can be displayed more appropriately. 【0170】 Furthermore, the optical flow calculation unit 20 determines the preceding image such that the feature point distance is greater than or equal to a predetermined distance for at least one of the two or more optical flows related to the image to be calculated, when the distance between the first feature point and the second feature point is defined as the feature point distance. 【0171】 According to this, for at least one of two or more optical flows related to the image to be calculated, the distance between the first feature point and the second feature point can be set to a predetermined distance or greater to calculate with greater accuracy. As a result, the vanishing point related to the image to be calculated can be calculated with greater accuracy, the pose of the moving object 1 can be estimated with greater accuracy, and the content can be displayed more appropriately. 【0172】 Furthermore, a posture estimation method according to one aspect of the present disclosure includes: a posture estimation step (step S4) having: a position information acquisition step of acquiring position information indicating the position of a vanishing point calculated based on two or more optical flows for each of a plurality of images captured by an imaging unit 2 mounted on a mobile body 1 and performing repeated imaging; a determination step of determining whether each of the plurality of vanishing points satisfies predetermined conditions based on each of the plurality of position information acquired in the position information acquisition step; and an estimation step of estimating the posture of the mobile body 1 based on one or more vanishing points that have been determined to satisfy predetermined conditions in the determination step; a superposition position determination step (step S5) of determining the superposition position of content to be displayed superimposed on an object based on the posture of the mobile body 1 estimated in the posture estimation step; and a display step (step S6) of displaying the content so as to be superimposed as a virtual image at the superposition position determined in the superposition position determination step. 【0173】 According to this, it will produce the same effects as the display system 10 described above. 【0174】 (Other embodiments, etc.) Although one or more embodiments of a display system, etc., have been described above based on the embodiments, this disclosure is not limited to these embodiments. Various modifications to the embodiments that a person skilled in the art could conceive of may also be included within the scope of this disclosure, as long as they do not deviate from the spirit of this disclosure. 【0175】 In the embodiments described above, each component may be implemented by dedicated hardware or by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU (Central Processing Unit) or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software that implements the apparatus of the embodiments described above is a program that causes a computer to execute each step included in the flowcharts shown in Figures 2 and 3. 【0176】 The following cases are also included in this disclosure. 【0177】 (1) The at least one device described above is specifically a computer system consisting of a microprocessor, ROM, RAM, hard disk unit, display unit, keyboard, mouse, etc. A computer program is stored in the RAM or hard disk unit. The at least one device described above achieves its function by the operation of the microprocessor in accordance with the computer program. Here, the computer program is composed of a combination of multiple instruction codes that indicate instructions to the computer in order to achieve a predetermined function. 【0178】 (2) Some or all of the components constituting at least one of the above-described devices may be made up of a single system LSI (Large Scale Integration). The system LSI is a multi-functional LSI manufactured by integrating multiple components onto a single chip, and specifically, it is a computer system comprising a microprocessor, ROM, RAM, etc. The RAM stores a computer program. The system LSI achieves its function by operating the microprocessor in accordance with the computer program. 【0179】 (3) Some or all of the components constituting at least one of the above-described devices may consist of an IC card or a standalone module that is detachable from the device. The IC card or module is a computer system consisting of a microprocessor, ROM, RAM, etc. The IC card or module may include the above-described multi-function LSI. The IC card or module achieves its function by the operation of the microprocessor in accordance with a computer program. The IC card or module may be tamper-resistant. 【0180】 (4) The disclosure may also be the methods described above. Alternatively, it may be a computer program that implements these methods using a computer, or a digital signal consisting of a computer program. 【0181】 Furthermore, this disclosure may also refer to a computer program or digital signal recorded on a computer-readable recording medium, such as a flexible disk, hard disk, CD (Compact Disc)-ROM, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray® Disc), semiconductor memory, etc. Alternatively, it may refer to a digital signal recorded on such a recording medium. 【0182】 Furthermore, this disclosure may also include the transmission of computer programs or digital signals via telecommunications lines, wireless or wired communication lines, networks such as the Internet, data broadcasting, etc. 【0183】 Alternatively, the program or digital signal may be carried out by another independent computer system by recording and transferring it on a recording medium, or by transferring the program or digital signal via a network or the like. 【0184】 (Note) Based on the above description of embodiments, the following technologies are disclosed. 【0185】 (Technology 1) An attitude estimation device comprising: a position information acquisition unit that acquires position information indicating the position of a vanishing point calculated based on two or more optical flows for each of a plurality of images captured by an imaging unit mounted on a moving body and performing repeated imaging; a determination unit that determines whether each of the plurality of vanishing points satisfies predetermined conditions based on each of the plurality of position information acquired by the position information acquisition unit; and an estimation unit that estimates the attitude of the moving body based on one or more vanishing points among the plurality of vanishing points that the determination unit has determined to satisfy the predetermined conditions, A superposition position determination device that determines the superposition position of content to be superimposed and displayed on the object based on the orientation of the moving object estimated by the orientation estimation device, The system includes a display device that displays the content as a virtual image superimposed on the superimposed position determined by the superimposed position determination device, Display system. 【0186】 (Technology 2) The aforementioned predetermined conditions include the first condition, The first condition is that, when the vanishing point subject to determination of whether or not the first condition is met is designated as the vanishing point to be determined, and the vanishing point of the image captured immediately before the image relating to the vanishing point to be determined among the plurality of images is designated as the comparison vanishing point, the amount of displacement of the vanishing point to be determined from the comparison vanishing point is within a predetermined range. The display system described in Technology 1. 【0187】 (Technology 3) The aforementioned predetermined conditions include the first condition, The first condition is that, when the vanishing point subject to determination of whether or not the first condition is met is designated as the vanishing point to be determined, and the image relating to the vanishing point to be determined among the plurality of images is designated as the image to be determined, the degree of variation of the two or more optical flows relating to the image to be determined with respect to the vanishing point to be determined is within a predetermined range. The display system described in Technology 1. 【0188】 (Technology 4) The aforementioned predetermined conditions include the first condition, The first condition is that, when the vanishing point subject to determination as to whether or not the first condition is met is designated as the vanishing point to be determined, the vanishing point to be determined is located within a predetermined range. The display system described in Technology 1. 【0189】 (Technology 5) The predetermined conditions include a second condition that the vanishing point is not in an image captured immediately after the image in which the vanishing point is determined not to satisfy the first condition among the plurality of images, or that the vanishing point is not in two or more images that are consecutive to the image captured immediately after the image in which the vanishing point is determined not to satisfy the first condition among the plurality of images. A display system as described in any of technologies 2 to 4. 【0190】 (Technology 6) The predetermined conditions include a second condition that the vanishing point in two or more consecutive images among the plurality of images is not the vanishing point in the image immediately following the last image captured among two or more consecutive images which are determined not to satisfy the first condition, or that the vanishing point in two or more consecutive images among the plurality of images is not the vanishing point in two or more images which are consecutive images starting from the image immediately following the last image captured among two or more consecutive images which are determined not to satisfy the first condition. A display system as described in any one of the technical items 2 to 4. 【0191】 (Technology 7) The aforementioned predetermined conditions include the third condition, The third condition is that the amount of change in the velocity of the moving object during a predetermined period up to the time when the image relating to the vanishing point, which is the subject of the determination of whether or not the third condition is met among the plurality of images, is captured is within a predetermined range. A display system as described in any of the technologies 1 through 6. 【0192】 (Technology 8) The estimation unit estimates the attitude of the moving body based on two or more vanishing points out of the one or more vanishing points determined by the determination unit to satisfy the predetermined conditions, where the degree of variation among the two or more vanishing points is within a predetermined range. A display system as described in any of the technologies 1 through 7. 【0193】 (Technology 9) The image acquisition unit acquires the aforementioned multiple images, An optical flow calculation unit that calculates the two or more optical flows related to each of the plurality of images, The system further includes a vanishing point calculation unit that calculates the vanishing point for each of the plurality of images based on the two or more optical flows for each of the plurality of images. A display system as described in any of the technologies 1 through 8. 【0194】 (Technology 10) The optical flow calculation unit, when it has selected two or more images from the plurality of images to be used for the calculation of optical flows as the calculation target images, and selected images taken before the calculation target images as the prior images, determines the prior images for each of the two or more optical flows related to the calculation target images, extracts feature locations that appear in both the calculation target images and the determined prior images, calculates a straight line passing through a first feature point in the calculation target image that indicates the position of the feature location in the calculation target image and a second feature point in the calculation target image that indicates the position of the feature location in the determined prior image, and sets the calculated straight line as the optical flow. The display system described in Technical 9. 【0195】 (Technology 11) The optical flow calculation unit determines, for at least one of the two or more optical flows related to the image to be calculated, that an image captured two or more times before the image to be calculated is the previous image. The display system described in Technical 10. 【0196】 (Technology 12) The optical flow calculation unit determines, for each of the two or more optical flows related to the image to be calculated, that an image captured two or more times before the image to be calculated is the previous image, where 50% or more of the optical flows are captured. The display system described in Technical 11. 【0197】 (Technology 13) The optical flow calculation unit determines the previous image such that, when the distance between the first feature point and the second feature point is defined as the feature point distance, the feature point distance for at least one of the two or more optical flows relating to the image to be calculated is greater than or equal to a predetermined distance. A display system as described in any of Technical 10 to 12. 【0198】 (Technology 14) A posture estimation step comprising: a position information acquisition step of acquiring position information indicating the position of a vanishing point calculated based on two or more optical flows relating to each of a plurality of images captured by an imaging unit mounted on a moving body that performs repeated imaging; a determination step of determining whether each of the plurality of vanishing points satisfies a predetermined condition based on each of the plurality of position information acquired by the position information acquisition step; and an estimation step of estimating the posture of the moving body based on one or more vanishing points among the plurality of vanishing points that are determined to satisfy the predetermined condition by the determination step, A superposition position determination step, which determines the superposition position of content to be superimposed and displayed on the object, based on the orientation of the moving object estimated by the orientation estimation step, The system includes a display step in which the content is displayed as a virtual image superimposed on the superimposed position determined by the superimposed position determination step. Display method. [Industrial applicability] 【0199】 This disclosure can be used in display systems and the like that estimate the orientation of a moving object and display content based on the orientation of the moving object. [Explanation of symbols] 【0200】 1 Mobile Unit 2 Imaging Unit 10 Display Systems 12 Posture estimation device 14. Superimposition position determination device 16 Display device 18 Image acquisition unit 20 Optical Flow Calculation Unit 22 Vanishing point calculation section 24 Location information acquisition section 26 Judgment section 28 Estimation part 30 Corner of the door 32 Corner of a building 34,40 First Feature Point 38 straight line 36, 42, 46, 50 Second feature point 44,48,52 straight line 54,56 intersection 58, 60, 62 Optical Flow 64 Vanishing point

Claims

[Claim 1] An attitude estimation device comprising: a position information acquisition unit that acquires position information indicating the position of a vanishing point calculated based on two or more optical flows relating to each of a plurality of images captured by an imaging unit mounted on a moving body and performing repeated imaging; a determination unit that determines whether each of the plurality of vanishing points satisfies predetermined conditions based on each of the plurality of position information acquired by the position information acquisition unit; and an estimation unit that estimates the attitude of the moving body based on one or more vanishing points among the plurality of vanishing points that the determination unit has determined to satisfy the predetermined conditions, A superposition position determination device that determines the superposition position of content to be superimposed and displayed on the object based on the orientation of the moving object estimated by the orientation estimation device, The system includes a display device that displays the content as a virtual image superimposed at the superimposed position determined by the superimposed position determination device, The aforementioned predetermined conditions include the first condition, The first condition is that when the vanishing point to be determined as the vanishing point to be determined is designated as the vanishing point to be determined, and the vanishing point of the image captured immediately before the image relating to the vanishing point to be determined among the plurality of images is designated as the vanishing point to be compared, the amount of displacement of the vanishing point to be determined from the comparison vanishing point is within a predetermined range. The predetermined conditions include a second condition that the vanishing point relating to two or more consecutive images among the plurality of images is not the vanishing point relating to the image immediately following the last image captured among two or more consecutive images which are determined not to satisfy the first condition, or that the vanishing point relating to two or more consecutive images from two or more images which are determined not to satisfy the first condition, starting from the image immediately following the last image captured among two or more consecutive images. Display system. [Claim 2] The aforementioned predetermined conditions include the third condition, The third condition is that the amount of change in the velocity of the moving object during a predetermined period up to the time when the image relating to the vanishing point, which is the subject of the determination of whether or not the third condition is met among the plurality of images, is captured is within a predetermined range. The display system according to claim 1. [Claim 3] The estimation unit estimates the attitude of the moving body based on two or more vanishing points out of the one or more vanishing points determined by the determination unit to satisfy the predetermined conditions, where the degree of variation among the two or more vanishing points is within a predetermined range. The display system according to claim 1 or 2. [Claim 4] The image acquisition unit acquires the aforementioned multiple images, An optical flow calculation unit that calculates two or more optical flows related to each of the plurality of images, The system further includes a vanishing point calculation unit that calculates the vanishing point for each of the plurality of images based on the two or more optical flows for each of the plurality of images. The display system according to claim 1 or 2. [Claim 5] The optical flow calculation unit, when it has selected two or more images from the plurality of images to be used for the calculation of optical flows as the calculation target images, and selected images taken before the calculation target images as the prior images, determines the prior images for each of the two or more optical flows related to the calculation target images, extracts feature locations that appear in both the calculation target image and the determined prior images, calculates a straight line passing through a first feature point in the calculation target image that indicates the position of the feature location in the calculation target image and a second feature point in the calculation target image that indicates the position of the feature location in the determined prior images, and sets the calculated straight line as the optical flow. The display system according to claim 4. [Claim 6] The optical flow calculation unit determines, for at least one of the two or more optical flows related to the image to be calculated, that an image captured two or more times before the image to be calculated is the previous image. The display system according to claim 5. [Claim 7] An attitude estimation device comprising: a position information acquisition unit that acquires position information indicating the position of a vanishing point calculated based on two or more optical flows relating to each of a plurality of images captured by an imaging unit mounted on a moving body and performing repeated imaging; a determination unit that determines whether each of the plurality of vanishing points satisfies a predetermined condition based on each of the plurality of position information acquired by the position information acquisition unit; and an estimation unit that estimates the attitude of the moving body based on one or more vanishing points among the plurality of vanishing points that the determination unit has determined to satisfy the predetermined condition, A superposition position determination device that determines the superposition position of content to be superimposed and displayed on the object based on the orientation of the moving object estimated by the orientation estimation device, A display device that displays the content superimposed as a virtual image at the superimposed position determined by the superimposed position determination device, The image acquisition unit acquires the aforementioned multiple images, An optical flow calculation unit that calculates two or more optical flows related to each of the plurality of images, The system includes a vanishing point calculation unit that calculates the vanishing point for each of the plurality of images based on the two or more optical flows for each of the plurality of images, The optical flow calculation unit, when it has selected two or more images from the plurality of images that are the subject of optical flow calculation as the calculation target images, and selected images taken before the calculation target images as the prior images, determines the prior images for each of the two or more optical flows related to the calculation target images, extracts feature locations that appear in both the calculation target image and the determined prior images, calculates a straight line passing through a first feature point in the calculation target image that indicates the position of the feature location in the calculation target image and a second feature point in the calculation target image that indicates the position of the feature location in the determined prior image, and sets the calculated straight line as the optical flow. The optical flow calculation unit determines, for each of the two or more optical flows related to the image to be calculated, that an image captured two or more times before the image to be calculated is the previous image, where 50% or more of the optical flows are captured. Display system. [Claim 8] The optical flow calculation unit determines the previous image such that, when the distance between the first feature point and the second feature point is defined as the feature point distance, the feature point distance for at least one of the two or more optical flows relating to the image to be calculated is greater than or equal to a predetermined distance. The display system according to claim 5. [Claim 9] A posture estimation step comprising: a position information acquisition step of acquiring position information indicating the position of a vanishing point calculated based on two or more optical flows relating to each of a plurality of images captured by an imaging unit mounted on a moving body that repeatedly performs imaging; a determination step of determining whether each of the plurality of vanishing points satisfies a predetermined condition based on each of the plurality of position information acquired by the position information acquisition step; and an estimation step of estimating the posture of the moving body based on one or more vanishing points among the plurality of vanishing points that are determined to satisfy the predetermined condition by the determination step, A superposition position determination step, which determines the superposition position of content to be superimposed and displayed on the object, based on the orientation of the moving object estimated by the orientation estimation step, The system includes a display step in which the content is displayed as a virtual image superimposed on the superimposed position determined by the superimposed position determination step, The aforementioned predetermined conditions include the first condition, The first condition is that when the vanishing point to be determined as the vanishing point to be determined is designated as the vanishing point to be determined, and the vanishing point of the image captured immediately before the image relating to the vanishing point to be determined among the plurality of images is designated as the vanishing point to be compared, the amount of displacement of the vanishing point to be determined from the comparison vanishing point is within a predetermined range. The predetermined conditions include a second condition that the vanishing point relating to two or more consecutive images among the plurality of images is not the vanishing point relating to the image immediately following the last image captured among two or more consecutive images which are determined not to satisfy the first condition, or that the vanishing point relating to two or more consecutive images from two or more images which are determined not to satisfy the first condition, starting from the image immediately following the last image captured among two or more consecutive images. Display method.

Images