Image generation device and program

Abstract

To provide an image generating apparatus and a program therefor which can easily correct a rendering image to fit an entire celestial image.SOLUTION: The present invention is directed to an image generating apparatus 1 for generating a superimposed image by superimposing a rendering image simulating an architecture on a captured image obtained by capturing a scheduled construction site A. The apparatus has an image acquiring unit 11 for acquiring the captured image as an entire celestial image and the rendering image, a capturing position acquiring unit 12 for acquiring a capturing position P including height at which the captured image is captured, a superimposing position determination unit 13 for determining a superimposition position of the captured image and the rendering image based on the acquired capturing position P, an image converting unit 14 for converting the rendering image into the entire celestial image based on the determined superimposition position, and an image generating unit 15 for generating the superimposed image by superimposing the converted rendering image on the captured image.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an image generation device and a program.

Background Art

[0002] Conventionally, by creating an exterior view of a building planned for construction, it has been practiced to show the image after completion. Thus, showing the exterior view is useful in housing sales business and the like.

[0003] Recently, as an exterior view, showing the image after completion using a 3D rendering image (hereinafter, also simply referred to as a rendering image) has also been carried out. By using a rendering image, the exterior can be confirmed from various viewpoints. Further, by superimposing the rendering image on a real photograph image of the planned construction site, harmony with the neighboring scenery can be confirmed. For example, a construction image display method has been proposed in which a 3D image (exterior image 9) of a building is superimposed on a photographed image of the planned location of the building (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By making a real photograph image of the planned construction site into an omnidirectional image, it is possible to confirm all 360 degrees in all directions from a predetermined viewpoint. On the other hand, it is also necessary to correct the rendering image in accordance with the omnidirectional image. Therefore, it is preferable if the rendering image can be easily corrected in accordance with the omnidirectional image.

[0006] The present invention aims to provide an image generation device and program that can easily correct rendered images to match 360-degree images. [Means for solving the problem]

[0007] The present invention relates to an image generation device that generates a superimposed image by superimposing a rendering image that mimics a building onto an image captured of a construction site, and comprises: an image acquisition unit that acquires the image captured, which is a 360-degree spherical image, and the rendering image; an image position acquisition unit that acquires the image position including the height at which the image captured was taken; an image position determination unit that determines the superimposition position of the image captured and the rendering image based on the acquired image position; an image conversion unit that converts the rendering image into a 360-degree spherical image based on the determined superimposition position; and an image generation unit that generates a superimposed image by superimposing the image captured and the converted rendering image at the determined superimposition position.

[0008] Furthermore, the image generation device further comprises a reference point acquisition unit that acquires a predetermined reference point in the captured image, a feature point acquisition unit that acquires a feature point indicating the position in the rendered image corresponding to the acquired reference point, and an estimation unit that estimates the imaging position based on the acquired reference point and feature point, wherein the imaging position acquisition unit preferably acquires the estimated imaging position.

[0009] Furthermore, the image generation device further comprises a reference point acquisition unit that acquires predetermined reference points in the captured image, a location information acquisition unit that acquires the location of the acquired reference points on a map as location information, and an estimation unit that estimates the imaging position based on the acquired reference points and the location information, wherein the imaging position acquisition unit preferably acquires the estimated imaging position.

[0010] Furthermore, it is preferable that the image acquisition unit acquires an image-captured video including the captured image, and the image generation unit generates a superimposed video by superimposing a rendering video, which includes the rendering image and whose temporal length has been adjusted to match a preset temporal length of the image-captured video, onto the image-captured video.

[0011] Furthermore, the present invention relates to a program that causes a computer to function as an image generation device that generates a superimposed image by superimposing a rendering image that mimics a building onto an image taken of a planned construction site, wherein the computer functions as an image acquisition unit that acquires the image taken, which is a 360-degree spherical image, and the rendering image; an image position acquisition unit that acquires the image position including the height at which the image was taken; a superimposed position determination unit that determines the superimposed position of the image taken and the rendering image based on the acquired image position; an image conversion unit that converts the rendering image into a 360-degree spherical image based on the determined superimposed position; and an image generation unit that generates a superimposed image by superimposing the image taken and the converted rendering image at the determined superimposed position. [Effects of the Invention]

[0012] According to the present invention, it is possible to provide an image generation device and program that can easily correct a rendered image to match a 360-degree spherical image. [Brief explanation of the drawing]

[0013] [Figure 1] This is a schematic diagram showing the relationship between the imaging position of the captured image synthesized by the image generation device according to the first embodiment of the present invention and the planned construction site included in the image target. [Figure 2] This is a schematic diagram showing the superimposed image generated by the image generation device of the first embodiment. [Figure 3] This is a schematic diagram showing the captured image synthesized by the image generation device of the first embodiment. [Figure 4] This is a schematic diagram showing a rendered image synthesized by the image generation apparatus according to the first embodiment. [Figure 5] This is a block diagram showing the configuration of the image generation device according to the first embodiment. [Figure 6] This is a block diagram showing the configuration of an image generation device according to a second embodiment of the present invention. [Figure 7] This is a block diagram showing the configuration of an image generation device according to the third embodiment of the present invention. [Modes for carrying out the invention]

[0014] The image generation apparatus 1 and program according to each embodiment of the present invention will be described below with reference to Figures 1 to 7. First, in describing the image generation apparatus 1 and program according to each embodiment, the generated images (hereinafter also referred to as superimposed images) will be explained with reference to Figures 1 to 4.

[0015] The superimposed image is, for example, an image obtained by superimposing a rendering image of a building onto an image taken from imaging position P at the planned construction site A, as shown in Figures 1 and 2. Specifically, the superimposed image is an image obtained by superimposing a rendering image of the planned building onto a real-world image of the planned construction site A. By superimposing the rendering image onto the real-world image, it is possible to check the harmony between the planned building and adjacent buildings, etc.

[0016] In the following implementations, a 360-degree spherical image is used as the captured image, as shown in Figure 3. This allows for a full view of the surrounding area from a predetermined viewpoint. Therefore, it is possible to check the harmony not only with adjacent buildings, but also with the surrounding environment.

[0017] On the one hand, the rendering image is, for example, a 3D model as shown in FIG. 4. The rendering image is produced using, for example, a mesh and a texture. Therefore, the rendering image needs to be transformed so that it can be superimposed on the omnidirectional image from a predetermined viewpoint. The image generation device 1 according to each of the following embodiments performs the transformation of the rendering image based on the imaging position P including the height. Thereby, the rendering image can be easily corrected according to the omnidirectional image.

[0018] [First Embodiment] Next, the image generation device 1 and the program according to the first embodiment of the present invention will be described with reference to FIGS. 1 to 5. The image generation device 1 is, for example, a device that acquires and superimposes an imaging image and a rendering image. As shown in FIG. 5, the image generation device 1 includes an image acquisition unit 11, an imaging position acquisition unit 12, a superimposition positioning unit 13, an image conversion unit 14, an image generation unit 15, and an output unit 16.

[0019] The image acquisition unit 11 is realized, for example, by the operation of the CPU. The image acquisition unit 11 acquires an imaging image, which is an omnidirectional image, and a rendering image. The image acquisition unit 11 acquires, as the imaging image, for example, an omnidirectional image captured by the user and an omnidirectional image captured from the imaging position P where the planned construction site A can be visually recognized. Further, the image acquisition unit 11 acquires, as the rendering image, for example, an image previously produced as a 3D model of a building.

[0020] The imaging position acquisition unit 12 is realized, for example, by the operation of the CPU. The imaging position acquisition unit 12 acquires the imaging position P including the height at which the imaging image is captured. The imaging position acquisition unit 12 acquires, for example, the latitude, longitude, and height (altitude) as the imaging position P. In the present embodiment, the imaging position acquisition unit 12 acquires the imaging position P input using an input device (not shown) such as a keyboard.

[0021] The superimposition position determination unit 13 is implemented, for example, by the operation of the CPU. The superimposition position determination unit 13 determines the superimposition positions of the captured image and the rendered image based on the acquired imaging position P. For example, the superimposition position determination unit 13 determines the position of the planned construction site A included in the captured image as the superimposition position of the rendered image with the imaging position P as the viewpoint. The superimposition position determination unit 13 also determines feature points of the rendered image to be superimposed on the reference points of the planned construction site A. For example, the superimposition position determination unit 13 uses multiple points at the boundary positions of the planned construction site A with the road as reference points and determines feature points of the rendered image corresponding to the reference points. That is, the superimposition position determination unit 13 determines feature points corresponding to the reference points from among multiple points at the boundary positions with the road included in the rendered image.

[0022] The image conversion unit 14 is implemented, for example, by the operation of the CPU. The image conversion unit 14 converts the rendered image into a 360-degree spherical image based on the determined superposition position. The image conversion unit 14 converts the rendered image with the imaging position P as the viewpoint into a 360-degree spherical image, for example, by positioning it so that the position of the reference point and the position of the feature point overlap.

[0023] The image generation unit 15 is implemented, for example, by the operation of the CPU. The image generation unit 15 generates a superimposed image by superimposing the captured image and the converted rendering image at a determined superposition position. The image generation unit 15 generates a superimposed image by superimposing the captured image and the converted rendering image, for example, by aligning the positions of feature points with the positions of reference points.

[0024] The output unit 16 is implemented, for example, by the operation of the CPU. The output unit 16 outputs the generated superimposed image to an external device. The output unit 16 displays the superimposed image on a display device (not shown), for example, a display. The output unit 16 also outputs (transmits) the superimposed image to another terminal, for example.

[0025] Next, the operation of the image generation device 1 according to this embodiment will be described. First, the image acquisition unit 11 acquires the captured image and the rendered image. Next, the imaging position acquisition unit 12 acquires the imaging position P of the captured image. Then, the superposition position determination unit 13 determines the superposition position of the rendered image on the captured image.

[0026] Next, the image conversion unit 14 converts the rendered image into a 360-degree spherical image based on the determined superposition position. Then, the image generation unit 15 superimposes the captured image and the converted rendered image to generate a superimposed image. Finally, the output unit 16 outputs the generated superimposed image.

[0027] Next, the program according to this embodiment will be described. Each component included in the image generation device 1 can be realized by hardware, software, or a combination thereof. Here, realization by software means that it is realized by a computer loading and executing a program.

[0028] Programs can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memory (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (random access memory)). Display programs may also be supplied to a computer using various types of transient computer-readable media. Examples of transient computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable media can be supplied to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0029] The image generation apparatus 1 and program according to this embodiment provide the following effects. (1) An image generation device 1 that generates a superimposed image by superimposing a rendering image of a building onto an image taken of a planned construction site A, comprising: an image acquisition unit 11 that acquires an image taken of a 360-degree spherical image and a rendering image; an image acquisition unit 12 that acquires an image acquisition position P including the height at which the image was taken; an image superimposition position determination unit 13 that determines the superimposition position of the image taken of the image taken of the image and the rendering image based on the acquired image acquisition position P; an image conversion unit 14 that converts the rendering image into a 360-degree spherical image based on the determined superimposition position; and an image generation unit 15 that generates a superimposed image by superimposing the image taken of the image and the converted rendering image at the determined superimposition position. Furthermore, the program causes a computer to function as an image generation device 1 that generates a superimposed image by superimposing a rendering image that mimics a building onto an image taken of a planned construction site A, and the computer functions as an image acquisition unit 11 that acquires an image taken of a 360-degree spherical image and a rendering image, an image position acquisition unit 12 that acquires an image position P including the height at which the image was taken, a superimposed position determination unit 13 that determines the superimposed position of the image taken and the rendering image based on the acquired image position P, an image conversion unit 14 that converts the rendering image into a 360-degree spherical image based on the determined superimposed position, and an image generation unit 15 that generates a superimposed image by superimposing the image taken and the converted rendering image at the determined superimposed position. This allows for easy correction of the rendered image to match the 360-degree image. In other words, the rendered image can be easily corrected to match the captured image. Therefore, a superimposed image can be easily obtained by superimposing the captured image and the rendered image. Furthermore, since the captured image and the rendered image are superimposed based on the imaging position P, which includes height, a superimposed image with less distortion can be obtained. In addition, since the reference point and feature point are associated using the height information of the imaging position, the accuracy of the superimposition can be improved compared to cases where only latitude and longitude are available and height information is not.

[0030] [Second Embodiment] Next, an image generation device 1 and program according to a second embodiment of the present invention will be described with reference to Figure 6. In describing the second embodiment, the same reference numerals are used for components that are the same as those in the previously described embodiment, and their descriptions are omitted or simplified. The image generation device 1 and program according to the second embodiment acquire reference points and feature points, and estimate the imaging position P. Furthermore, the image generation device 1 according to the second embodiment differs from the first embodiment in that the imaging position acquisition unit 12 acquires the estimated imaging device. As shown in Figure 6, the image generation device 1 according to the second embodiment further comprises a reference point acquisition unit 17, a feature point acquisition unit 18, and an estimation unit 19.

[0031] The reference point acquisition unit 17 is implemented, for example, by the operation of the CPU. The reference point acquisition unit 17 acquires predetermined reference points in the captured image. The reference point acquisition unit 17 acquires, for example, the position of a reference point in the captured image. The reference point acquisition unit 17 acquires reference points by receiving a specification of the position in the captured image. In this embodiment, the reference point acquisition unit 17 acquires multiple points of the boundary with the road and the boundary with the neighboring property as reference points.

[0032] The feature point acquisition unit 18 is implemented, for example, by the operation of the CPU. The feature point acquisition unit 18 acquires feature points that indicate the position in the rendered image, corresponding to the acquired reference points. The feature point acquisition unit 18 acquires feature points, for example, by accepting a specification of the position in the rendered image.

[0033] The estimation unit 19 is implemented, for example, by the operation of the CPU. The estimation unit 19 estimates the imaging position P based on the acquired reference points and feature points. The estimation unit 19 estimates the imaging position P, including height, from the position of the reference points and the position of the corresponding feature points. The estimation unit 19 estimates the imaging position P from the degree of deviation between the position of the reference points and the position of the corresponding feature points.

[0034] Next, the operation of the image generation device 1 according to this embodiment will be described. First, the image acquisition unit 11 acquires the captured image and the rendered image. Next, the reference point acquisition unit 17 acquires the reference point. The feature point acquisition unit 18 acquires the feature point. Then, the estimation unit 19 estimates the imaging position P, including height, from the acquired reference point and feature point. The imaging position acquisition unit 12 acquires the imaging position P of the captured image.

[0035] Next, the superposition position determination unit 13 determines the superposition position of the rendered image relative to the captured image. Then, the image conversion unit 14 converts the rendered image into a 360-degree spherical image based on the determined superposition position. Next, the image generation unit 15 superimposes the captured image and the converted rendered image to generate a superimposed image. Finally, the output unit 16 outputs the generated superimposed image.

[0036] The image generation apparatus 1 and program according to this embodiment provide the following effects. (2) The image generation device 1 further includes a reference point acquisition unit 17 that acquires predetermined reference points in the captured image, a feature point acquisition unit 18 that acquires feature points indicating the position in the rendered image corresponding to the acquired reference points, and an estimation unit 19 that estimates the imaging position P based on the acquired reference points and feature points, and the imaging position acquisition unit 12 acquires the estimated imaging position P. As a result, even if the imaging position P cannot be acquired, the imaging position P can be estimated if the reference points and feature points can be obtained. Therefore, the sense of incongruity in the superposition of the captured image and the rendered image can be suppressed.

[0037] [Third Embodiment] Next, an image generation device 1 and program according to the third embodiment of the present invention will be described with reference to Figure 7. In describing the third embodiment, the same reference numerals are used for components identical to those in the previously described embodiments, and their descriptions are omitted or simplified. The image generation device 1 and program according to the third embodiment estimate the imaging position P, including height, by acquiring a reference point and accepting the specification of an imaging point on a map. Furthermore, the image generation device 1 according to the third embodiment differs from the first embodiment in that the imaging position acquisition unit 12 acquires the estimated imaging device. The image generation device 1 according to the third embodiment differs from the second embodiment in that it further includes a location information acquisition unit 20, as shown in Figure 7. Furthermore, the image generation device 1 according to the third embodiment differs from the second embodiment in that it does not include a feature point acquisition unit 18. Furthermore, the image generation device 1 according to the third embodiment differs from the second embodiment in that the estimation unit 19 estimates the imaging position P based on the acquired reference point and location information.

[0038] The location information acquisition unit 20 is implemented, for example, by the operation of the CPU. The location information acquisition unit 20 acquires the location of the acquired reference point on the map as location information. The location information acquisition unit 20 acquires location information, for example, using EXIF ​​(Exchangeable image file format) contained in the captured image.

[0039] Next, the operation of the image generation device 1 according to this embodiment will be described. First, the image acquisition unit 11 acquires the captured image and the rendered image. Next, the reference point acquisition unit 17 acquires the reference point. The location information acquisition unit 20 acquires location information. Then, the estimation unit 19 estimates the imaging position P, including height, from the acquired reference point and location information. The imaging position acquisition unit 12 acquires the imaging position P of the captured image.

[0040] Next, the superposition position determination unit 13 determines the superposition position of the rendered image relative to the captured image. Then, the image conversion unit 14 converts the rendered image into a 360-degree spherical image based on the determined superposition position. Next, the image generation unit 15 superimposes the captured image and the converted rendered image to generate a superimposed image. Finally, the output unit 16 outputs the generated superimposed image.

[0041] The image generation apparatus 1 and program according to this embodiment provide the following effects. (3) The image generation device 1 further includes a reference point acquisition unit 17 that acquires predetermined reference points in the captured image, a location information acquisition unit 20 that acquires the location of the acquired reference points on a map as location information, and an estimation unit 19 that estimates the imaging position P based on the acquired reference points and location information, and the imaging position acquisition unit 12 acquires the estimated imaging position P. As a result, even if the height of the imaging position P cannot be acquired, the height of the imaging position P can be estimated if the reference points and location information can be obtained. Therefore, the sense of incongruity in the superposition of the captured image and the rendered image can be suppressed.

[0042] Although preferred embodiments of the image generation apparatus and program of the present disclosure have been described above, the present disclosure is not limited to the embodiments described above and can be modified as appropriate. For example, in the above embodiment, the imaging position acquisition unit 12 acquires the imaging position P by input from an input device, but it is not limited to this. For example, the imaging position acquisition unit 12 may acquire the imaging position P from an imaging position storage unit (not shown) that stores the imaging position P in advance. Alternatively, the imaging position acquisition unit 12 may acquire the imaging position P from an external terminal or the like.

[0043] Furthermore, in the second or third embodiment described above, the reference point acquisition unit 17, the feature point acquisition unit 18, and the location information acquisition unit 20 may acquire reference points, feature points, or location information input by the input device. In addition, the reference point acquisition unit 17 and the feature point acquisition unit 18 may acquire reference points or feature points from the edge position (position with a large spatial frequency).

[0044] Furthermore, although the above embodiment described captured images and rendered images as examples, a captured video including a captured image and a rendered captured image including a rendered image may be superimposed. The image acquisition unit 11 acquires a captured video including a captured image. The image generation unit 15 generates a superimposed video by superimposing a rendered video including a rendered image, whose temporal length has been adjusted to match a preset temporal length of the captured video, onto the captured video. Here, the image generation device 1 may further include a time acquisition unit (not shown) that acquires the time (for example, disclosure time and end time) used for superimposition from the acquired captured video. The image generation device 1 may also further include a rendering video generation unit (not shown) that adjusts or generates a rendered video according to the acquired time. [Explanation of symbols]

[0045] 1. Image generation device 11 Image acquisition unit 12 Imaging position acquisition unit 13. Superposition position determination unit 14 Image conversion unit 15 Image generation unit 17 Reference point acquisition section 18 Feature Point Acquisition Unit 19 Estimation part 20 Location Information Acquisition Unit A: Planned construction site P imaging position

Claims

1. An image generation device that generates a superimposed image by superimposing a 2D image, which is a 3D model of a virtual object onto an image of a target area, An image acquisition unit that acquires the captured image, which is a 360-degree image, and the 3D model, A unit for acquiring imaging position acquires the imaging position including the height at which the aforementioned image was captured, An overlay position determination unit determines the overlay position of the captured image and the two-dimensional image obtained by converting the three-dimensional model, based on the acquired imaging position. An image conversion unit converts the three-dimensional model into a 360-degree spherical image based on the determined superposition position to obtain the two-dimensional image, An image generation unit generates a superimposed image by superimposing the captured image and the converted two-dimensional image at a determined superposition position, A reference point acquisition unit that acquires a predetermined reference point in the captured image, A feature point acquisition unit acquires feature points that indicate the position in the three-dimensional model corresponding to the acquired reference points, An estimation unit that estimates the imaging position based on the acquired reference points and feature points, Equipped with, The aforementioned imaging position acquisition unit is an image generation device that acquires the estimated imaging position.

2. An image generation device that generates a superimposed image by superimposing a 2D image, which is a 3D model of a virtual object onto an image of a target area, An image acquisition unit that acquires the captured image, which is a 360-degree image, and the 3D model, A unit for acquiring imaging position acquires the imaging position including the height at which the aforementioned image was captured, A superposition position determination unit determines the superposition position of the captured image and the two-dimensional image obtained by converting the three-dimensional model, based on the acquired imaging position. An image conversion unit converts the three-dimensional model into a 360-degree spherical image based on the determined superposition position to obtain the two-dimensional image, An image generation unit generates a superimposed image by superimposing the captured image and the converted two-dimensional image at a determined superposition position, A reference point acquisition unit that acquires a predetermined reference point in the captured image, A location information acquisition unit acquires the location of the acquired reference point on a map as location information, An estimation unit that estimates the imaging position based on the acquired reference point and location information, Equipped with, The aforementioned imaging position acquisition unit is an image generation device that acquires the estimated imaging position.

3. This program causes a computer to function as an image generation device that generates a superimposed image by superimposing a 2D image, which is a 3D model of a virtual object onto an image of a target area, The aforementioned computer, An image acquisition unit that acquires the captured image, which is a 360-degree spherical image, and the 3D model. A unit that acquires the imaging position including the height at which the aforementioned image was captured, A superposition position determination unit determines the superposition position of the captured image and the two-dimensional image obtained by converting the three-dimensional model, based on the acquired imaging position. An image conversion unit converts the three-dimensional model into a 360-degree spherical image based on the determined superposition position to obtain the two-dimensional image. An image generation unit generates a superimposed image by superimposing the captured image and the converted two-dimensional image at a determined superposition position. A reference point acquisition unit that acquires a predetermined reference point in the captured image, A feature point acquisition unit acquires feature points that indicate the position in the three-dimensional model corresponding to the acquired reference points. An estimation unit estimates the imaging position based on the acquired reference points and feature points. To make it function as, The aforementioned imaging position acquisition unit is a program that acquires the estimated imaging position.

4. This program causes a computer to function as an image generation device that generates a superimposed image by superimposing a 2D image, which is a 3D model of a virtual object onto an image of a target area, The aforementioned computer, An image acquisition unit that acquires the captured image, which is a 360-degree spherical image, and the 3D model. A unit that acquires the imaging position including the height at which the aforementioned image was captured, A superposition position determination unit determines the superposition position of the captured image and the two-dimensional image obtained by converting the three-dimensional model, based on the acquired imaging position. An image conversion unit converts the three-dimensional model into a 360-degree spherical image based on the determined superposition position to obtain the two-dimensional image. An image generation unit generates a superimposed image by superimposing the captured image and the converted two-dimensional image at a determined superposition position. A reference point acquisition unit that acquires a predetermined reference point in the captured image, A location information acquisition unit acquires the location of the acquired reference point on a map as location information. An estimation unit estimates the imaging position based on the acquired reference point and location information. To make it function as, The aforementioned imaging position acquisition unit is a program that acquires the estimated imaging position.

Images