AR image display device and AR image display program

Abstract

To provide an AR image display device and an AR image display program capable of suppressing deviation between a road surface image and three-dimensional point group data in an AR image.SOLUTION: An AR image display device 1 includes: a holding unit 11 that holds three-dimensional point group data P2; an acquisition unit 12 that acquires a road surface image P1 in a travel direction of a moving object M; a display unit 13 that configures and displays an AR image P3b by superimposing the three-dimensional point group data P2 on the road surface image P1 based on a current position LN of the moving object M; and an adjusting unit 14 that adjusts a relation between the road surface image P1 and the three-dimensional point group data P2 in the AR image P3b.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an AR video display device and an AR video display program. [Background technology]

[0002] Patent Document 1 describes a display device. This display device controls the display so that the virtual image superimposed on the superimposition target is maintained as seen by the driver, even when a pitch change occurs in the host vehicle. That is, when a pitch change occurs in the host vehicle, the vertical positional relationship between the superimposition target, the projection position of the image onto the transparent member, and the driver's eye point changes. If the projection position of the image onto the transparent member is maintained, the virtual image appears to be shifted from the superimposition target as seen by the driver. Therefore, the display device corrects the projection position of the image onto the transparent member in accordance with a pitch change in the host vehicle so as to reduce the display shift of the virtual image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-030917 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, the display device according to Patent Document 1 displays a virtual image directly on a transparent member such as a head-up display. In addition, in Feature Point 1, the virtual image is exemplified by a virtual image of an arrow indicating a right or left turn, etc.

[0005] On the other hand, according to the inventor's knowledge, there is a demand for displaying AR images generated by superimposing 3D point cloud data (obtained in advance, for example, by a mobile mapping system, etc.) showing the road surface and structures around the road onto road surface images obtained by a camera on a mobile terminal mounted on a moving object, for purposes such as supporting visibility by installing it on a snow removal vehicle or inspecting road infrastructure.

[0006] In this case, there is no problem when the moving object is traveling on a flat road, but when the moving object is traveling on an inclined road, there is a risk of a discrepancy in consistency between the road surface image and the 3D point cloud data. As a specific example, when the moving object is traveling uphill, the 3D point cloud data may appear to be floating in relation to the road surface image, and when the moving object is traveling downhill, the 3D point cloud data may appear to be sinking in relation to the road surface image.

[0007] Therefore, an object of the present invention is to provide an AR image display device and an AR image display program that can suppress the deviation between a road surface image and three-dimensional point cloud data in an AR image. [Means for solving the problem]

[0008] The AR image display device of the present invention is [1] "an AR image display device mounted on a mobile body moving on a road surface, comprising: a storage unit that stores 3D point cloud data indicating structures in a target area including the road surface; an acquisition unit that acquires a road surface image that is an image of the road surface in the traveling direction of the mobile body; a display unit that creates and displays an AR image by superimposing the 3D point cloud data on the road surface image based on the current position of the mobile body; and an adjustment unit that adjusts the relationship between the road surface image in the AR image and the 3D point cloud data, wherein the adjustment unit has: a first elevation acquisition unit that acquires a first elevation value that is the elevation at the current position of the mobile body; a second elevation acquisition unit that acquires a second elevation value that is the elevation at a position forward or backward a set distance from the current position along the traveling direction; an inclination angle acquisition unit that acquires the inclination angle of the road surface based on the set distance, the first elevation value, and the second elevation value; and a pitch angle adjustment unit that adjusts the pitch angle when superimposing the 3D point cloud data on the road surface image based on the inclination angle."

[0009] This device displays an AR (augmented reality) image constructed by superimposing 3D point cloud data showing structures in a target area, including the road surface on which the mobile object moves, onto a road surface image. In particular, this device acquires the inclination angle of the road surface based on a first elevation value, which is the elevation at the current position of the mobile object, a second elevation value, which is the elevation at a position forward or backward a set distance from the current position, and the set distance. Then, based on the inclination angle, the pitch angle used when superimposing the 3D point cloud data on the road surface image is adjusted. Therefore, in the AR image, deviation between the road surface image and the 3D point cloud data according to the inclination of the road surface is suppressed.

[0010] The AR image display device according to the present invention may be [2] "the AR image display device according to the above [1], wherein the acquisition unit acquires the road surface image using a camera of a mobile terminal mounted on the moving body, and the display unit displays the AR image on a display of the mobile terminal." In this case, it becomes possible to display the AR image on the mobile terminal mounted on the moving body while suppressing deviation between the road surface image and the 3D point cloud data.

[0011] The AR image display device according to the present invention may be [3] "the AR image display device according to the above [2], in which the gyro sensor of the mobile terminal is turned off." In this case, the influence of vibrations of the moving object or the like is eliminated.

[0012] The AR image display device according to the present invention may be [4] "the AR image display device according to any one of [1] to [3] above, wherein the first altitude acquisition unit acquires the current position based on a GNSS signal and acquires the first altitude value by referring to altitude data stored in advance based on the current position, and the second altitude acquisition unit acquires the second altitude value by referring to the altitude data based on the forward position or the backward position." In this case, it is possible to easily and reliably acquire a highly accurate altitude value.

[0013] The AR image display device according to the present invention may be [5] "the AR image display device according to the above [4], in which the altitude data is an altitude tile." In this case, it is possible to more easily and reliably obtain a highly accurate altitude value.

[0014] The AR image display device according to the present invention may be [6] "the AR image display device according to any one of [1] to [5] above, including an input receiving unit that receives input of the set distance." In this case, it is possible to arbitrarily set the position where the second altitude value is acquired.

[0015] The AR image display program of the present invention is [7] "an AR image display program for causing a computer mounted on a mobile body moving on a road surface to function as an AR image display device, the program realizing in the computer a storage function for storing 3D point cloud data showing structures in a target area including the road surface, an acquisition function for acquiring a road surface image that is an image of the road surface in the traveling direction of the mobile body, a display function for constructing and displaying an AR image by superimposing the 3D point cloud data on the road surface image based on the current position of the mobile body, and an adjustment function for adjusting the relationship between the road surface image and the 3D point cloud data in the AR image, the adjustment function including a first elevation acquisition function for acquiring a first elevation value that is the elevation at the current position of the mobile body, a second elevation acquisition function for acquiring a second elevation value that is the elevation at a position forward or backward a set distance from the current position along the traveling direction, and an inclination angle acquisition function for acquiring an inclination angle of the road surface based on the set distance, the first elevation value, and the second elevation value, and a pitch angle adjustment function that adjusts the pitch angle when the 3D point cloud data is superimposed on the road surface image based on the inclination angle.

[0016] This program displays AR video in which 3D point cloud data indicating structures in a target area, including the road surface on which the mobile object moves, is superimposed on road surface video. In particular, this program acquires the inclination angle of the road surface based on a first elevation value, which is the elevation at the current position of the mobile object, a second elevation value, which is the elevation at a position forward or backward a set distance from the current position, and the set distance. Then, based on the inclination angle, the pitch angle used when superimposing the 3D point cloud data on the road surface video is adjusted. Therefore, in the AR video, deviation between the road surface video and the 3D point cloud data according to the inclination of the road surface is suppressed. [Effects of the Invention]

[0017] According to the present invention, it is possible to provide an AR image display device and an AR image display program that can suppress deviation between a road surface image and three-dimensional point cloud data in an AR image. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 is a schematic diagram showing a moving object equipped with a mobile terminal including an AR image display device according to this embodiment. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of the AR video display device configured in the mobile terminal shown in FIG. [Figure 3] FIG. 3 is a diagram showing an example of a road surface image acquired by an acquisition unit at a predetermined position. [Figure 4] FIG. 4 is a diagram showing an example of three-dimensional point cloud data at the same position as the road surface image in FIG. [Figure 5] FIG. 5 is a diagram showing an example of an AR image generated by superimposing the three-dimensional point cloud data of FIG. 4 on the road surface image of FIG. [Figure 6] FIG. 6 is a flowchart illustrating an example of the operation of the adjustment unit. [Figure 7] FIG. 7 is a flowchart showing an example of a method for obtaining altitude values. [Figure 8] FIG. 8 is a schematic diagram showing an example of an elevation tile. [Figure 9] FIG. 9 is a diagram for explaining a method for calculating the tilt angle. [Figure 10] FIG. 10 is a diagram showing an example of the adjusted AR image. [Figure 11] FIG. 11 is a diagram showing the AR video display program according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0019] An embodiment will be described below with reference to the drawings. In the description of each drawing, the same or corresponding elements are designated by the same reference numerals, and redundant description may be omitted.

[0020] Fig. 1 is a schematic diagram showing a moving body equipped with a mobile terminal including an AR image display device according to this embodiment. Fig. 2 is a block diagram showing the functional configuration of the AR image display device configured in the mobile terminal shown in Fig. 1. As shown in Figs. 1 and 2, the AR image display device 1 is configured in a mobile terminal A mounted on a moving body M moving on a road surface S. The road surface S is, for example, the surface of a road or a railroad track. In addition, the moving body M is, for example, a vehicle such as an industrial vehicle, such as a snowplow.

[0021] The mobile terminal A includes a rear camera A1, a display A2, a GNSS receiver A3, and a gyro sensor A4. The mobile terminal A is mounted on a moving body M so that the rear camera A1 is on the outside of the moving body M (i.e., on the road surface S side) and the display A2 is on the inside of the moving body M (i.e., on the passenger side of the moving body M).

[0022] The rear camera A1 captures an image of a target area including a road surface S. The display A2 displays the AR image generated by the AR image display device 1. The GNSS receiver A3 receives radio waves transmitted from GNSS satellites to acquire the current position of the mobile terminal A (i.e., moving body M). The gyro sensor A4 detects the angular velocity of the mobile terminal A (i.e., moving body M) to detect shaking occurring in the mobile terminal A (i.e., moving body M).

[0023] Mobile terminal A is physically configured as a computer system including a CPU, a main storage device configured with memories such as RAM and ROM, an auxiliary storage device configured with a hard disk and memory, a communication control device, etc. Each functional unit of the AR image display device 1 can be realized by loading specific computer software onto the hardware of mobile terminal A, such as the CPU and main storage device, to operate the communication control device, etc. under the control of the CPU and to read and write data from and to the main storage device and auxiliary storage device.

[0024] The AR image display device 1 includes a storage unit 11, an acquisition unit 12, a display unit 13, an adjustment unit 14, and an input reception unit 15. The storage unit 11 stores three-dimensional point cloud data indicating structures in a target area including a road surface S. The three-dimensional point cloud data is acquired in advance, for example, by a mobile mapping system or the like, and is data in which three-dimensional coordinates are assigned to each point. The acquisition unit 12 acquires a road surface image, which is an image of the road surface S in the traveling direction of a moving object M, for example, via a rear camera A1 of a mobile terminal A. The display unit 13 creates an AR image by superimposing the three-dimensional point cloud data on the road surface image acquired by the acquisition unit 12 based on the current position of the moving object M, and displays the AR image on, for example, a display A2 of the mobile terminal A.

[0025] Fig. 3 is a diagram showing an example of a road surface image at a predetermined position. The acquisition unit 12 acquires the road surface image P1 shown in Fig. 3 as an example. As shown in Fig. 3, the road surface image P1 includes a road surface S and various structures D in a target area including the road surface S. Examples of the structures D include a dividing line D1 drawn on the road surface S, a tree D2 installed on the side of the road surface S, a sign D3 on the road surface S, an outdoor lamp D4 installed on the side of the road surface S, and a crosswalk D5 drawn on the road surface S.

[0026] Fig. 4 is a diagram showing an example of three-dimensional point cloud data at the same position as the road surface image of Fig. 3. The storage unit 11 stores, as an example, three-dimensional point cloud data P2 shown in Fig. 4. As shown in Fig. 4, the three-dimensional point cloud data P2 includes a point cloud corresponding to a road surface S and a point cloud corresponding to various structures D in a target area including the road surface S. Examples of the point clouds corresponding to the structures D include a point cloud E1 corresponding to a dividing line D1 drawn on the road surface S, a point cloud E2 corresponding to a tree D2 planted on the side of the road surface S, a point cloud E3 corresponding to a sign D3 on the road surface S, a point cloud E4 corresponding to an outdoor lamp D4 installed in the side strip of the road surface S, and a point cloud E5 corresponding to a crosswalk D5 drawn on the road surface S.

[0027] Fig. 5 is a diagram showing an example of an AR image generated by superimposing the 3D point cloud data of Fig. 4 on the road surface image of Fig. 3. AR image P3a shown in Fig. 5 shows a structure D and a point cloud corresponding to the structure D. This AR image P3a does not take into account changes in the pitch angle of the mobile terminal A (i.e., the moving object M) according to the inclination of the road surface S. For this reason, it can be seen that in AR image P3a, each point cloud of the 3D point cloud data P2 is superimposed on each structure D in the road surface image P1 with a shift according to the inclination of the road surface S. For example, in area R of AR image P3a, point cloud E1 corresponding to demarcation line D1 is superimposed on the demarcation line D1 in a floating manner.

[0028] To eliminate such a discrepancy, the adjustment unit 14 adjusts the relationship between the road surface image P1 in the AR image P3a and the 3D point cloud data P2. Here, an outline of the operation of the adjustment unit 14 will be described, and the detailed operation will be described later. The adjustment unit 14 includes a first elevation acquisition unit 21, a second elevation acquisition unit 22, a tilt angle acquisition unit 23, and a pitch angle adjustment unit 24.

[0029] The first altitude acquisition unit 21 acquires a first altitude value, which is the altitude at the current position of the mobile terminal A (i.e., the moving body M). The second altitude acquisition unit 22 acquires a second altitude value, which is the altitude at a position forward or backward (here, the forward position) a set distance from the current position in the traveling direction of the moving body M. The set distance may be set to a predetermined value in advance, or may be set by the input receiving unit 15 receiving an input from the user.

[0030] The inclination angle acquisition unit 23 acquires the inclination angle of the road surface S based on the set distance, the first altitude value, and the second altitude value. The pitch angle adjustment unit 24 adjusts the pitch angle when superimposing the 3D point cloud data P2 on the road surface image P1 based on the inclination angle. This generates an AR image (see FIG. 10) in which the deviation between each structure D in the road surface image P1 and each point cloud in the 3D point cloud data P2 is eliminated. The display unit 13 displays this adjusted AR image.

[0031] Next, detailed operation of the adjustment unit 14 will be described. Fig. 6 is a flowchart showing an example of the operation of the adjustment unit. As shown in Fig. 6, the adjustment unit 14 first acquires a set distance TD (step S101). The adjustment unit 14 may acquire the set distance TD (see Figs. 8 and 9) that is set in advance and stored in the storage unit 11 by accessing the storage unit 11, or may acquire the set distance TD as an input value from the user via the input receiving unit 15.

[0032] Next, the first altitude acquisition unit 21 acquires the current position LN (see FIGS. 8 and 9) of the moving object M based on a GNSS (Global Navigation Satellite System) signal acquired via the GNSS receiver A3 of the mobile terminal A (step S102). The GNSS signal can be received, for example, via the GNSS antenna B shown in FIG. 1. The first altitude acquisition unit 21 may also use a high-precision positioning system (for example, an RTK (Real Time Kinematic) system) to acquire the current position LN with higher accuracy. As an example, the RTK-GNSS system makes it possible to acquire the current position with an error of several centimeters.

[0033] Next, the first altitude acquisition unit 21 acquires a first altitude value, which is an altitude value at the current position LN, based on the current position LN acquired in step S102 (step S103). The method for acquiring the altitude value will be described in more detail below.

[0034] FIG. 7 is a flowchart showing an example of a method for acquiring elevation values. FIG. 8 is a schematic diagram showing an example of an elevation tile. The storage unit 11 stores an elevation tile Q shown in FIG. 8. As shown in FIGS. 7 and 8, the first elevation acquisition unit 21 acquires the tile coordinates of the elevation tile Q corresponding to the current position LN (position coordinates, for example, latitude and longitude) acquired in step S102 (step S201). This identifies the tile CT corresponding to the current position LN among the elevation tiles Q. An elevation value of 256 × 256 pixels is set in one tile CT (see the Geospatial Information Authority of Japan website (https: / / maps.gsi.go.jp / development / demtile.html)). Next, the first elevation acquisition unit 21 acquires the pixel coordinates corresponding to the current position LN among the tiles CT identified in step S201 (step S202).

[0035] This identifies one pixel in the tile CT that corresponds to the current position LN. Then, the first elevation acquisition unit 21 acquires the elevation tile value of the pixel (step S203). The first elevation acquisition unit 21 sets the elevation tile value acquired in step S203 as the first elevation value. In this way, the first elevation acquisition unit 21 acquires the current position LN based on the GNSS signal, and also acquires the first elevation value by referring to the elevation tile Q based on the current position LN.

[0036] Note that the first elevation acquisition unit 21 is not limited to referring to the elevation tile Q stored in the storage unit 11, and may refer to the elevation tile Q stored in any storage device accessible via the communication device of the mobile terminal A. In this case, the storage unit 11 does not need to store the elevation tile Q.

[0037] Continuing with reference to Fig. 6, in the next step, the second altitude acquisition unit 22 acquires a forward position LA (see Figs. 8 and 9) (or a backward position (same below)), which is a position a set distance TD ahead of the current position LN in the traveling direction of the moving body M (step S104). Then, the second altitude acquisition unit 22 acquires a second altitude value, which is the altitude value at the forward position LA, based on the forward position 104 acquired in step S104 (step S105). In step S105, the second altitude value can be acquired by a method similar to that of step S103 described above.

[0038] 7 and 8, the second elevation acquisition unit 22 acquires the tile coordinates of the elevation tile Q corresponding to the forward position LA (position coordinates, for example, latitude and longitude) acquired in step S104 (step S201). This identifies the tile CT corresponding to the forward position LA among the elevation tiles Q. Next, the second elevation acquisition unit 22 acquires the pixel coordinates corresponding to the forward position LA among the tiles CT identified in step S201 (step S202).

[0039] This identifies one pixel in the tile CT that corresponds to the forward position LA. Then, the second elevation acquisition unit 22 acquires the elevation tile value of the pixel (step S203). The second elevation acquisition unit 22 sets the elevation tile value acquired in step S203 as the second elevation value. In this way, the second elevation value is acquired by referring to the elevation tile Q based on the forward position LA.

[0040] Continuing with reference to FIG. 6, in the next step, the inclination angle acquisition unit 23 calculates the inclination angle of the road surface S based on the set distance TD, the first altitude value, and the second altitude value (step S106). More specifically, as shown in FIG. 9, the inclination angle acquisition unit 23 calculates the altitude difference TH between the current position LN and the forward position LA using the difference between the first altitude value acquired in step S103 and the second altitude value acquired in step S105. Then, the inclination angle acquisition unit 23 calculates tan -1 The inclination angle θ of the road surface S is calculated by calculating (altitude difference TH / set distance TD).

[0041] Then, in the next step, the pitch angle adjustment unit 24 adjusts the pitch angle when superimposing the 3D point cloud data P2 on the road surface image P1 based on the tilt angle θ acquired in step S106 (step S107). That is, in step S107, the pitch angle adjustment unit 24 adjusts the viewing frustum pitch angle of the AR display in accordance with the tilt angle θ.

[0042] As a result, an AR image P3b is generated in which the misalignment between each structure D in the road surface image P1 and each point cloud in the 3D point cloud data P2 is eliminated, as shown in Figure 10. Referring to area R as an example, in AR image P3b, the point cloud E1 corresponding to the demarcation line D1 in the pre-adjustment AR image P3a is eliminated from the superimposed points, making the demarcation line D1 and the point cloud E1 corresponding to the demarcation line D1 coincide with each other. The display unit 13 displays the adjusted AR image P3b generated as described above on the display A2 of the mobile terminal A.

[0043] The adjustment of the road surface image P1 and the 3D point cloud data P2 and the display of the AR image P3b can be performed continuously in real time while the moving object M is moving (operating). This allows the occupants of the moving object M to receive visibility support during snow removal work, road infrastructure inspections, etc., while referring to the AR image P3b displayed in real time. The AR image display device 1 can turn off the gyro sensor A4 of the mobile terminal A to eliminate the effects of vibrations and the like that accompany the movement of the moving object M.

[0044] 11, an AR image display program for causing a computer (mobile terminal A) mounted on a moving object M moving on a road surface S to function as an AR image display device 1 will be described. The AR image display program AP includes a main module m10, a holding module m11, an acquisition module m12, a display module m13, an adjustment module m14, and an input reception module m15.

[0045] The main module m10 is a part that performs overall control. The functions (storage function, acquisition function, display function, adjustment function) realized by executing the storage module m11, acquisition module m12, display module m13, adjustment module m14, and input reception module m15 are similar to the storage unit 11, acquisition unit 12, display unit 13, adjustment unit 14, and input reception unit 15 of the AR video display device 1 shown in FIG. 1, respectively.

[0046] The adjustment module m14 includes a first elevation acquisition module m21, a second elevation acquisition module m22, a tilt angle acquisition module m23, and a pitch angle adjustment module m24. The functions (first elevation acquisition function, second elevation acquisition function, tilt angle acquisition function, and pitch angle adjustment function) realized by executing the first elevation acquisition module m21, the second elevation acquisition module m22, the tilt angle acquisition module m23, and the pitch angle adjustment module m24 are similar to the first elevation acquisition unit 21, the second elevation acquisition unit 22, the tilt angle acquisition unit 23, and the pitch angle adjustment unit 24 of the adjustment unit 14 shown in FIG.

[0047] The above-described AR video display program AP is provided by a storage medium M1 (e.g., a main storage device or an auxiliary storage device), such as a magnetic disk, an optical disk, or a semiconductor memory. The AR video display program AP may also be provided via a communication network as a computer data signal superimposed on a carrier wave.

[0048] As described above, the AR image display device 1 according to this embodiment displays AR image P3b configured by superimposing 3D point cloud data P2 indicating structures D in a target area including a road surface S along which a moving object M moves on a road surface image P1. In particular, the AR image display device 1 acquires an inclination angle θ of the road surface S based on a first elevation value that is the elevation at the current position LN of the moving object M, a second elevation value that is the elevation at a position forward or backward (here, the forward position) from the current position LN by a set distance TD, and the set distance TD. Then, based on the inclination angle θ, a pitch angle used when superimposing the 3D point cloud data P2 on the road surface image P1 is adjusted. Thus, in the AR image P3b, deviation between the road surface image P1 and the 3D point cloud data P2 according to the inclination of the road surface S is suppressed.

[0049] Furthermore, in the AR image display device 1 according to this embodiment, the acquisition unit 12 acquires a road surface image P1 using the rear camera A1 of the mobile terminal A mounted on the moving body M, and the display unit 13 displays the AR image P3b on the display A2 of the mobile terminal A. Therefore, on the mobile terminal A mounted on the moving body M, it is possible to display the AR image P3b while suppressing deviation between the road surface image P1 and the three-dimensional point cloud data P2.

[0050] Furthermore, the AR image display device 1 according to this embodiment can turn off the gyro sensor A4 of the mobile terminal A. In this case, the influence of vibrations of the moving object M and the like is eliminated.

[0051] Furthermore, in the AR image display device 1 according to this embodiment, the first altitude acquisition unit 21 acquires the current position LN based on the GNSS signal and acquires the first altitude value by referring to the altitude tile Q based on the current position LN, and the second altitude acquisition unit 22 acquires the second altitude value by referring to the altitude tile Q based on the forward position LA. This makes it possible to easily and reliably acquire the altitude value.

[0052] Furthermore, the AR image display device 1 according to this embodiment includes an input receiving unit 15 that receives input of the set distance TD, which makes it possible to arbitrarily set the position where the first altitude value is obtained.

[0053] The above embodiment describes one aspect of the AR video display device and the AR video display program AP according to the present invention. Therefore, the AR video display device 1 and the AR video display program AP according to the present invention are not limited to the above embodiment and may be modified as desired.

[0054] For example, in the above embodiment, a case has been described in which 3D point cloud data P2 including a point cloud corresponding to structures (i.e., structures present on the road surface S) D in a target area including the road surface S is superimposed on the road surface image P1. However, the AR image display device 1 and the AR image display program AP may also superimpose data indicating buried objects (e.g., pipes) buried under the road surface S on the road surface image P1. In this case, when superimposing the data on the road surface image P1, the adjustment unit 14 can adjust the pitch angle of the data in accordance with the inclination angle θ of the road surface S, similar to the case in which the 3D point cloud data P2 is superimposed.

[0055] In the above embodiment, the case has been described in which the acquisition unit 12 acquires road surface images using the rear camera A1 of the mobile terminal A. However, the acquisition unit 12 is not limited to a camera integrally built into the mobile terminal A, such as the rear camera A1 of the mobile terminal A, and may acquire road surface images using any camera that can acquire images using the mobile terminal A, such as an external camera connected to the mobile terminal A by wire or wirelessly. In other words, the camera of the mobile terminal A includes a camera integrally built into the mobile terminal A, a camera connected to the mobile terminal A, and the like.

[0056] Furthermore, turning off gyro sensor A4 in the above embodiment is not limited to turning off all gyro sensor A4, but also includes using the values ​​of some specific axes of gyro sensor A4 while not using the values ​​of the remaining axes (turning them off).

[0057] Furthermore, in the above embodiment, the first elevation acquisition unit 21 acquires the first elevation value and the second elevation value by referring to the elevation tile Q. However, the first elevation acquisition unit 21 may acquire the first elevation value and the second elevation value by referring to any elevation data stored in advance in the storage unit 11 or in a storage device accessible via a communication device of the mobile terminal A, instead of (or in addition to) the elevation tile Q. An example of elevation data other than the elevation tile is, for example, an image with location information in GeoTiff format. [Explanation of symbols]

[0058] 1...AR image display device, 11...holding unit, 12...acquisition unit, 13...display unit, 14...adjustment unit, 21...first altitude acquisition unit, 22...second altitude acquisition unit, 23...tilt angle acquisition unit, 24...pitch angle adjustment unit, A...mobile terminal, A1...rear camera (camera), A2...display.

Claims

1. An AR image display device mounted on a moving body moving on a road surface, a storage unit that stores three-dimensional point cloud data representing structures in a target area including the road surface; an acquisition unit that acquires a road surface image that is an image of the road surface in the traveling direction of the moving object; a display unit that creates and displays an AR image by superimposing the three-dimensional point cloud data on the road surface image based on the current position of the moving object; an adjustment unit that adjusts the relationship between the road surface image in the AR image and the three-dimensional point cloud data; Equipped with The adjustment unit a first altitude acquisition unit that acquires a first altitude value that is the altitude at a current position of the moving object; a second altitude acquisition unit that acquires a second altitude value that is an altitude at a position forward or backward a set distance from the current position along the traveling direction; an inclination angle acquisition unit that acquires an inclination angle of the road surface based on the set distance, the first altitude value, and the second altitude value; a pitch angle adjustment unit that adjusts a pitch angle when the three-dimensional point cloud data is superimposed on the road surface image based on the tilt angle; having AR image display device.

2. the acquisition unit acquires the road surface image using a camera of a mobile terminal mounted on the moving body, The display unit displays the AR image on a display of the mobile terminal. The AR image display device according to claim 1 .

3. Turning off the gyro sensor of the mobile terminal; The AR image display device according to claim 2 .

4. the first altitude acquisition unit acquires the current position based on a GNSS signal, and acquires the first altitude value by referring to altitude data stored in advance based on the current position; the second altitude acquisition unit acquires the second altitude value by referring to the altitude data based on the forward position or the rearward position. The AR image display device according to claim 1 .

5. The elevation data is an elevation tile. The AR image display device according to claim 4 .

6. an input receiving unit that receives an input of the set distance; The AR image display device according to any one of claims 1 to 5.

7. An AR image display program for causing a computer mounted on a mobile object moving on a road surface to function as an AR image display device, The computer, a storage function for storing three-dimensional point cloud data representing structures in a target area including the road surface; an acquisition function for acquiring a road surface image, which is an image of the road surface in the traveling direction of the moving body; a display function for constructing and displaying an AR image by superimposing the three-dimensional point cloud data on the road surface image based on the current position of the moving object; an adjustment function for adjusting the relationship between the road surface image in the AR image and the three-dimensional point cloud data; To achieve this, The adjustment function is a first altitude acquisition function for acquiring a first altitude value that is the altitude at the current position of the moving object; a second altitude acquisition function that acquires a second altitude value that is the altitude at a position forward or backward a set distance from the current position along the traveling direction; an inclination angle acquisition function that acquires an inclination angle of the road surface based on the set distance, the first altitude value, and the second altitude value; a pitch angle adjustment function that adjusts a pitch angle when the three-dimensional point cloud data is superimposed on the road surface image based on the inclination angle; Including, AR video display program.

Images