Image generating apparatus and program
Patent Information
- Application Number
- JP2025016892
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-02-04
- Publication Date
- 2025-06-02
- Estimated Expiration
- 2040-11-19
AI Technical Summary
Existing methods struggle to easily correct rendered images to match spherical images of construction sites, limiting the ability to confirm harmony from all directions.
An image generation device and program that acquires both captured and rendered images as sphere images, determines the superimposed position based on the imaging position, converts the rendered image to match the spherical image, and generates a superimposed image for easy correction.
Enables easy correction of rendered images to align with spherical images, allowing for accurate confirmation of building harmony from all directions, reducing discomfort in the superimposed image.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an image generating device and a program. [Background technology]
[0002] Conventionally, an exterior drawing of a building to be constructed has been prepared to show an image of what the building will look like after completion. Showing an exterior drawing is useful in sales of housing.
[0003] Nowadays, 3D rendering images (hereinafter simply referred to as rendering images) are often used as exterior views to show the image of the completed building. By using rendering images, the exterior can be confirmed from various viewpoints. Furthermore, by superimposing a rendering image on an actual image of the proposed construction site, it is possible to confirm the harmony with the surrounding scenery. For example, a construction image display strategy has been proposed in which a 3D image (exterior image 9) of the building is superimposed on an image taken of the proposed construction site (see, for example, Patent Document 1). [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2014-89697 A Summary of the Invention [Problem to be solved by the invention]
[0005] By converting an actual image of a planned construction site into a spherical image, it is possible to check 360 degrees in all directions from a specific viewpoint. On the other hand, the rendering image also needs to be corrected to match the spherical image. Therefore, it would be preferable if the rendering image could be easily corrected to match the spherical image.
[0006] An object of the present invention is to provide an image generating device and a program capable of easily correcting a rendering image to match a spherical image. [Means for solving the problem]
[0007] The present invention relates to an image generating device that generates a superimposed image by superimposing a rendering image that resembles a building on a captured image of a planned construction site, the image generating device including: an image acquiring unit that acquires the captured image, which is a celestial sphere image, and the rendering image; an imaging position acquiring unit that acquires an imaging position including a height at which the captured image was captured; a superimposition position determining unit that determines a superimposition position of the captured image and the rendering image based on the acquired imaging position; an image converting unit that converts the rendering image into a celestial sphere image based on the determined superimposition position; and an image generating unit that generates the superimposed image by superimposing the captured image and the converted rendering image at the determined superimposition position.
[0008] In addition, it is preferable that the image generating device further includes a reference point acquisition unit that acquires a predetermined reference point in the captured image, a feature point acquisition unit that acquires a feature point that indicates a position in the rendering image that corresponds to the acquired reference point, and an estimation unit that estimates the imaging position based on the acquired reference point and feature point, and the imaging position acquisition unit acquires the estimated imaging position.
[0009] In addition, it is preferable that the image generating device further includes a reference point acquisition unit that acquires a specific reference point in the captured image, a location information acquisition unit that acquires the position of the acquired reference point on a map as location information, and an estimation unit that estimates the imaging position based on the acquired reference point and the location information, and the imaging position acquisition unit acquires the estimated imaging position.
[0010] It is also preferable that the image acquisition unit acquires a captured moving image including the captured image, and the image generation unit superimposes a rendering moving image including the rendering image, the temporal length of which has been adjusted to match a predetermined temporal length of the captured moving image, onto the captured moving image to generate a superimposed moving image.
[0011] The present invention also relates to a program that causes a computer to function as an image generating device that generates a superimposed image by superimposing a rendering image that resembles a building on a captured image of a planned construction site, the program causing the computer to function as an image acquiring unit that acquires the captured image, which is a celestial sphere image, and the rendering image, an imaging position acquiring unit that acquires an imaging position including a height at which the captured image was captured, a superimposition position determining unit that determines a superimposition position of the captured image and the rendering image based on the acquired imaging position, an image converting unit that converts the rendering image into a celestial sphere image based on the determined superimposition position, and an image generating unit that generates the superimposed image by superimposing the captured image and the converted rendering image at the determined superimposition position. Effect of the Invention
[0012] According to the present invention, it is possible to provide an image generating device and a program capable of easily correcting a rendering image to match a spherical image. [Brief description of the drawings]
[0013] [Figure 1] 1 is a schematic diagram showing a relationship between an imaging position of a captured image synthesized by an image generating device according to a first embodiment of the present invention and a planned construction site included in an imaging target. [Diagram 2] FIG. 2 is a schematic diagram showing a superimposed image superimposed by the image generating device of the first embodiment. [Diagram 3] 1 is a schematic diagram showing a captured image synthesized by the image generating device of the first embodiment. FIG. [Figure 4] 2 is a schematic diagram showing a rendering image synthesized by the image generating device according to the first embodiment. FIG. [Diagram 5] 1 is a block diagram showing a configuration of an image generating device according to a first embodiment. [Figure 6] FIG. 11 is a block diagram showing a configuration of an image generating device according to a second embodiment of the present invention. [Figure 7] FIG. 11 is a block diagram showing a configuration of an image generating device according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] An image generating device 1 and a program according to each embodiment of the present invention will be described below with reference to FIGS. First, in describing the image generating device 1 and the program according to each embodiment, a generated image (hereinafter, also referred to as a superimposed image) will be described with reference to FIGS. 1 to 4. FIG.
[0015] The superimposed image is, for example, an image in which a rendering image of a building is superimposed on a captured image of the planned construction site A taken from an imaging position P, as shown in Figures 1 and 2. Specifically, the superimposed image is an image in which a rendering image of the building to be built is superimposed on a real-life image of the planned construction site A. By superimposing the rendering image on the real-life image, it is possible to confirm the harmony between the building to be built and neighboring buildings, etc.
[0016] In the following embodiments, a spherical image is used as the captured image, as shown in Fig. 3. This allows you to check the entire circumference from a specific viewpoint. Therefore, you can check the harmony with the surrounding environment, not just the neighboring buildings.
[0017] On the other hand, the rendering image is, for example, a 3D model as shown in FIG. 4. The rendering image is created using, for example, a mesh and a texture. Therefore, the rendering image needs to be converted so that it can be superimposed on a celestial sphere image from a predetermined viewpoint. The image generating device 1 according to each of the following embodiments converts the rendering image based on an imaging position P including a height. This makes it possible to easily correct the rendering image to match the celestial sphere image.
[0018] [First embodiment] Next, an image generating device 1 and a program according to a first embodiment of the present invention will be described with reference to FIGS. The image generating device 1 is, for example, a device that acquires a captured image and a rendered image and superimposes them. As shown in Fig. 5, the image generating device 1 includes an image acquiring unit 11, an imaging position acquiring unit 12, a superimposition position determining unit 13, an image converting unit 14, an image generating unit 15, and an output unit 16.
[0019] The image acquisition unit 11 is realized by, for example, the operation of a CPU. The image acquisition unit 11 acquires a captured image, which is a celestial sphere image, and a rendering image. The image acquisition unit 11 acquires, as the captured image, for example, a celestial sphere image captured by a user from an imaging position P where the planned construction site A can be viewed. In addition, the image acquisition unit 11 acquires, as the rendering image, for example, an image created in advance as a 3D model of a building.
[0020] The imaging position acquisition unit 12 is realized, for example, by the operation of a CPU. The imaging position acquisition unit 12 acquires an imaging position P including the height at which the captured image is captured. The imaging position acquisition unit 12 acquires, for example, latitude, longitude, and height (altitude) as the imaging position P. In this embodiment, the imaging position acquisition unit 12 acquires the imaging position P inputted using an input device (not shown) such as a keyboard.
[0021] The superimposition position determination unit 13 is realized by, for example, the operation of a CPU. The superimposition position determination unit 13 determines the superimposition position of the captured image and the rendering image based on the acquired imaging position P. For example, the superimposition position determination unit 13 determines the position of the planned construction site A included in the captured image as the superimposition position of the rendering image with the imaging position P as the viewpoint. In addition, the superimposition position determination unit 13 determines a feature point of the rendering image to be superimposed on a reference point of the planned construction site A. For example, the superimposition position determination unit 13 determines a feature point of the rendering image corresponding to a reference point by using a plurality of points of the boundary position with the road of the planned construction site A as the reference point. That is, the superimposition position determination unit 13 determines a feature point corresponding to the reference point among a plurality of points of the boundary position with the road included in the rendering image.
[0022] The image conversion unit 14 is realized, for example, by the operation of a CPU. The image conversion unit 14 converts the rendering image into a celestial sphere image based on the determined superimposition position. The image conversion unit 14 converts the rendering image, whose viewpoint is the imaging position P, into a celestial sphere image, so that the rendering image is arranged at a position where the position of the reference point and the position of the feature point overlap each other, for example.
[0023] The image generating unit 15 is realized, for example, by the operation of a CPU. The image generating unit 15 generates a superimposed image by superimposing the captured image and the converted rendering image at a determined superimposition position. The image generating unit 15 generates the superimposed image by superimposing the captured image and the converted rendering image, for example, so that the position of the feature point is superimposed on the position of the reference point.
[0024] The output unit 16 is realized, for example, by the operation of a CPU. The output unit 16 outputs the generated superimposed image to the outside. The output unit 16 displays the superimposed image on a display device (not shown), for example, such as a display. The output unit 16 also outputs (transmits) the superimposed image to, for example, another terminal or the like.
[0025] Next, the operation of the image generating device 1 according to this embodiment will be described. First, the image acquisition unit 11 acquires a captured image and a rendered image. Next, the imaging position acquisition unit 12 acquires an imaging position P of the captured image. Next, the superimposition position determination unit 13 determines a superimposition position of the rendering image relative to the captured image.
[0026] Next, the image conversion unit 14 converts the rendering image into a spherical image based on the determined superimposition position. Next, the image generation unit 15 generates a superimposed image by superimposing the captured image and the converted rendering image. Next, the output unit 16 outputs the generated superimposed image.
[0027] Next, the program according to this embodiment will be described. Each component included in the image generating device 1 can be realized by hardware, software, or a combination of these. Here, being realized by software means being realized by a computer reading and executing a program.
[0028] The program can be stored and provided to the computer using various types of non-transitory computer readable media. The non-transitory computer readable media includes various types of tangible storage media. Examples of the non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)). The display program may also be provided to the computer by various types of transitory computer readable media. Examples of the transitory computer readable media include electric signals, optical signals, and electromagnetic waves. The transitory computer readable media can provide the program to the computer via a wired communication path such as an electric wire and an optical fiber, or via a wireless communication path.
[0029] The image generating device 1 and the program according to the present embodiment described above provide the following advantages. (1) An image generating device 1 that generates a superimposed image by superimposing a rendering image that resembles a building on a captured image of a planned construction site A, and includes an image acquiring unit 11 that acquires a captured image that is a celestial sphere image and a rendering image, an imaging position acquiring unit 12 that acquires an imaging position P including the height at which the captured image was captured, a superimposition position determining unit 13 that determines a superimposition position of the captured image and the rendering image based on the acquired imaging position P, an image converting unit 14 that converts the rendering image into a celestial sphere image based on the determined superimposition position, and an image generating unit 15 that generates the superimposed image by superimposing the captured image and the converted rendering image at the determined superimposition position. The program also causes a computer to function as an image generating device 1 that generates a superimposed image by superimposing a rendering image that resembles a building on a captured image of a planned construction site A, and causes the computer to function as an image acquiring unit 11 that acquires a captured image that is a celestial sphere image and a rendering image, an imaging position acquiring unit 12 that acquires an imaging position P including the height at which the captured image was captured, a superimposition position determining unit 13 that determines a superimposition position of the captured image and the rendering image based on the acquired imaging position P, an image converting unit 14 that converts the rendering image into a celestial sphere image based on the determined superimposition position, and an image generating unit 15 that generates the superimposed image by superimposing the captured image and the converted rendering image at the determined superimposition position. This allows the rendering image to be easily corrected to match the spherical image. That is, the rendering image can be easily corrected to match the captured image. Therefore, a superimposed image in which the captured image and the rendering image are superimposed can be easily obtained. In addition, since the captured image and the rendering image are superimposed based on the imaging position P including the height, a superimposed image with less awkwardness can be obtained. In addition, since the reference point and the feature point are associated using information on the height of the imaging position, the accuracy of superimposition can be improved compared to a case where there is no information on the height of only the latitude and longitude.
[0030] [Second embodiment] Next, an image generating device 1 and a program according to a second embodiment of the present invention will be described with reference to FIG. 6. In describing the second embodiment, the same components as those in the above-described embodiment are given the same reference symbols, and their description will be omitted or simplified. The image generating device 1 and a program according to the second embodiment acquire a reference point and a feature point, and estimate an imaging position P. The image generating device 1 according to the second embodiment differs from the first embodiment in that an imaging position acquisition unit 12 acquires an estimated imaging device. As shown in FIG. 6, the image generating device 1 according to the second embodiment further includes a reference point acquisition unit 17, a feature point acquisition unit 18, and an estimation unit 19.
[0031] The reference point acquisition unit 17 is realized, for example, by the operation of a CPU. The reference point acquisition unit 17 acquires a predetermined reference point in the captured image. The reference point acquisition unit 17 acquires, for example, a position in the captured image that will be the reference point. The reference point acquisition unit 17 acquires the reference point by accepting a designation of a position in the captured image. In this embodiment, the reference point acquisition unit 17 acquires multiple points on the boundary with the road and the boundary with the neighbor's land as reference points.
[0032] The feature point acquisition unit 18 is realized, for example, by the operation of a CPU. The feature point acquisition unit 18 acquires feature points indicating positions in a rendering image corresponding to the acquired reference points. The feature point acquisition unit 18 acquires feature points, for example, by accepting designation of positions in a rendering image.
[0033] The estimation unit 19 is realized, for example, by the operation of a CPU. The estimation unit 19 estimates the imaging position P based on the acquired reference point and feature point. The estimation unit 19 estimates the imaging position P including the height, for example, from the position of the reference point and the position of the corresponding feature point. The estimation unit 19 estimates the imaging position P from the degree of deviation between the position of the reference point and the position of the corresponding feature point, for example.
[0034] Next, the operation of the image generating device 1 according to this embodiment will be described. First, the image acquisition unit 11 acquires a captured image and a rendered image. Next, the reference point acquisition unit 17 acquires a reference point. Furthermore, the feature point acquisition unit 18 acquires a feature point. Next, the estimation unit 19 estimates an imaging position P including a height from the acquired reference point and feature point. The imaging position acquisition unit 12 acquires the imaging position P of the captured image.
[0035] Next, the superimposition position determination unit 13 determines the superimposition position of the rendering image on the captured image. Next, the image conversion unit 14 converts the rendering image into a celestial sphere image based on the determined superimposition position. Next, the image generation unit 15 generates a superimposed image by superimposing the captured image and the converted rendering image. Next, the output unit 16 outputs the generated superimposed image.
[0036] The image generating device 1 and the program according to the present embodiment described above provide the following advantages. (2) The image generating device 1 further includes a reference point acquisition unit 17 that acquires a predetermined reference point in the captured image, a feature point acquisition unit 18 that acquires a feature point that indicates a position in the rendered image corresponding to the acquired reference point, and an estimation unit 19 that estimates the imaging position P based on the acquired reference point and feature point, and the imaging position acquisition unit 12 acquires the estimated imaging position P. As a result, even if the imaging position P cannot be acquired, the imaging position P can be estimated as long as the reference point and the feature point can be obtained. Therefore, it is possible to suppress the awkwardness of superimposing the captured image and the rendered image.
[0037] [Third embodiment] Next, an image generating device 1 and a program according to a third embodiment of the present invention will be described with reference to FIG. 7. In describing the third embodiment, the same components as those in the above-mentioned embodiment are denoted by the same reference symbols, and their description will be omitted or simplified. The image generating device 1 and the program according to the third embodiment estimate an imaging position P including a height by acquiring a reference point and accepting a designation of an imaging point on a map. The image generating device 1 according to the third embodiment differs from the first embodiment in that an imaging position acquisition unit 12 acquires an estimated imaging device. The image generating device 1 according to the third embodiment differs from the second embodiment in that it further includes a point information acquisition unit 20 as shown in FIG. 7. The image generating device 1 according to the third embodiment differs from the second embodiment in that it does not include a feature point acquisition unit 18. The image generating device 1 according to the third embodiment differs from the second embodiment in that an estimation unit 19 estimates an imaging position P based on the acquired reference point and point information.
[0038] The point information acquisition unit 20 is realized, for example, by the operation of a CPU. The point information acquisition unit 20 acquires the positions of the acquired reference points on a map as point information. The point information acquisition unit 20 acquires the point information by using, for example, EXIF (Exchangeable image file format) included in the captured image.
[0039] Next, the operation of the image generating device 1 according to this embodiment will be described. First, the image acquisition unit 11 acquires a captured image and a rendered image. Next, the reference point acquisition unit 17 acquires a reference point. Furthermore, the location information acquisition unit 20 acquires location information. Next, the estimation unit 19 estimates the imaging position P including the height from the acquired reference point and location information. The imaging position acquisition unit 12 acquires the imaging position P of the captured image.
[0040] Next, the superimposition position determination unit 13 determines the superimposition position of the rendering image on the captured image. Next, the image conversion unit 14 converts the rendering image into a celestial sphere image based on the determined superimposition position. Next, the image generation unit 15 generates a superimposed image by superimposing the captured image and the converted rendering image. Next, the output unit 16 outputs the generated superimposed image.
[0041] The image generating device 1 and the program according to the present embodiment described above provide the following advantages. (3) The image generating device 1 further includes a reference point acquisition unit 17 that acquires a predetermined reference point in the captured image, a point information acquisition unit 20 that acquires the position of the acquired reference point on the map as point information, and an estimation unit 19 that estimates the imaging position P based on the acquired reference point and point information, and the imaging position acquisition unit 12 acquires the estimated imaging position P. As a result, even if the height of the imaging position P cannot be acquired, the height of the imaging position P can be estimated as long as the reference point and point information can be obtained. Therefore, it is possible to suppress the awkwardness of superimposing the captured image and the rendering image.
[0042] Although preferred embodiments of the image generating device and program of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments and can be modified as appropriate. For example, in the above embodiment, the imaging position acquisition unit 12 acquires the imaging position P by input from an input device, but is not limited to this. For example, the imaging position acquisition unit 12 may acquire the imaging position P from an imaging position storage unit (not shown) that stores the imaging position P in advance. Also, the imaging position acquisition unit 12 may acquire the imaging position P from an external terminal or the like.
[0043] In the second or third embodiment, the reference point acquisition unit 17, the feature point acquisition unit 18, and the point information acquisition unit 20 may acquire the reference point, the feature point, or the point information input by an input device. The reference point acquisition unit 17 and the feature point acquisition unit 18 may acquire the reference point or the feature point from the edge position (the position with a large spatial frequency).
[0044] In the above embodiment, the captured image and the rendering image are described as examples, but the captured video including the captured image and the rendered captured image including the rendering image may be superimposed. The image acquisition unit 11 acquires the captured video including the captured image. The image generation unit 15 superimposes the rendered video including the rendering image, the time length of which is adjusted according to the time length of the captured video set in advance, on the captured video to generate a superimposed video. Here, the image generation device 1 may further include a time acquisition unit (not shown) that acquires the time (e.g., start time and end time) to be used for superimposition from among the acquired captured video. The image generation device 1 may further include a rendering video generation unit (not shown) that adjusts or generates the rendering video according to the acquired time. [Explanation of symbols]
[0045] 1 Image generation device 11 Image acquisition section 12 Imaging position acquisition unit 13 Superimposition position determination unit 14 Image conversion section 15 Image Generation Unit 17 Reference point acquisition section 18 Feature point acquisition unit 19 Estimation part 20 Location information acquisition unit A Construction site P Imaging position
Claims
1. An image generating device that generates a superimposed image by superimposing a two-dimensional image obtained by converting a three-dimensional model of a virtual object onto an image captured of a target area, an image acquisition unit that acquires the captured image, which is a spherical image, and the three-dimensional model; an imaging position acquisition unit that acquires an imaging position including a height at which the captured image is captured; a superimposition position determination unit that determines a superimposition position of the captured image and the two-dimensional image obtained by converting the three-dimensional model based on the acquired imaging position; an image conversion unit that converts the three-dimensional model into a spherical image based on the determined superimposition position to generate the two-dimensional image; an image generating unit that generates a superimposed image by superimposing the captured image and the converted two-dimensional image at a determined superimposition position; a reference point acquisition unit for acquiring a predetermined reference point in the captured image; a feature point acquisition unit that acquires feature points indicating positions in the three-dimensional model corresponding to the acquired reference points; an estimation unit that estimates the imaging position based on the acquired reference point and the acquired feature point; Equipped with The imaging position acquisition unit is an image generating device that acquires the estimated imaging position.
2. An image generating device that generates a superimposed image by superimposing a two-dimensional image obtained by converting a three-dimensional model that imitates a virtual object onto an image captured of a target area, an image acquisition unit that acquires the captured image, which is a spherical image, and the three-dimensional model; an imaging position acquisition unit that acquires an imaging position including a height at which the captured image is captured; a superimposition position determination unit that determines a superimposition position of the captured image and the two-dimensional image obtained by converting the three-dimensional model based on the acquired imaging position; an image conversion unit that converts the three-dimensional model into a spherical image based on the determined superimposition position to generate the two-dimensional image; an image generating unit that generates a superimposed image by superimposing the captured image and the converted two-dimensional image at a determined superimposition position; a reference point acquisition unit for acquiring a predetermined reference point in the captured image; a location information acquisition unit that acquires the location of the acquired reference point on a map as location information; an estimation unit that estimates the imaging position based on the acquired reference point and the spot information; Equipped with The imaging position acquisition unit is an image generating device that acquires the estimated imaging position.
3. The imaging target site is a construction site, and the virtual object is a construction target.
3. An image generating device according to claim 1 or 2.
4. A program that causes a computer to function as an image generating device that generates a superimposed image by superimposing a two-dimensional image obtained by converting a three-dimensional model that resembles a virtual object onto an image captured of a target area, The computer, an image acquisition unit that acquires the captured image, which is a spherical image, and the three-dimensional model; an imaging position acquisition unit that acquires an imaging position including a height at which the captured image is captured; a superimposition position determination unit that determines a superimposition position of the captured image and the two-dimensional image obtained by converting the three-dimensional model based on the acquired imaging position; an image conversion unit that converts the three-dimensional model into a spherical image based on the determined superimposition position to generate the two-dimensional image; an image generating unit that generates a superimposed image by superimposing the captured image and the converted two-dimensional image at a determined superimposition position; a reference point acquisition unit for acquiring a predetermined reference point in the captured image; a feature point acquisition unit that acquires feature points indicating positions in the three-dimensional model corresponding to the acquired reference points; an estimation unit that estimates the imaging position based on the acquired reference point and the acquired feature point; Function as a The imaging position acquisition unit is a program for acquiring the estimated imaging position.
5. A program that causes a computer to function as an image generating device that generates a superimposed image by superimposing a two-dimensional image obtained by converting a three-dimensional model that resembles a virtual object onto an image captured of a target area, The computer, an image acquisition unit that acquires the captured image, which is a spherical image, and the three-dimensional model; an imaging position acquisition unit that acquires an imaging position including a height at which the captured image is captured; a superimposition position determination unit that determines a superimposition position of the captured image and the two-dimensional image obtained by converting the three-dimensional model based on the acquired imaging position; an image conversion unit that converts the three-dimensional model into a spherical image based on the determined superimposition position to generate the two-dimensional image; an image generating unit that generates a superimposed image by superimposing the captured image and the converted two-dimensional image at a determined superimposition position; a reference point acquisition unit for acquiring a predetermined reference point in the captured image; a spot information acquisition unit that acquires the positions of the acquired reference points on a map as spot information; an estimation unit that estimates the imaging position based on the acquired reference point and the spot information; Function as a The imaging position acquisition unit is a program for acquiring the estimated imaging position.