Method for creating orthomosaic images, orthomosaic image creation system, method for creating 3D models, and 3D model creation system

Abstract

This invention provides an orthomosaic image creation method, a system for creating orthomosaic images based on images taken from above, a 3D model creation method, a 3D model creation system, and a marker used therein. [Solution] The method involves taking a first image of a road where part of the road surface is covered by an obstacle when viewed from above, using a first imaging device from an altitude higher than the obstacle; taking a second image of the area covered by the obstacle using a second imaging device from an altitude lower than the obstacle; obtaining the 3D coordinates of a first feature point included in the first image; obtaining the 3D coordinates of a second feature point included in the second image; and creating a corrected orthomosaic image by correcting at least a portion of the area of ​​the road surface covered by the obstacle to an area not covered by the obstacle based on the 3D coordinates of the first feature point and the 3D coordinates of the second feature point.

Description

Technical Field

[0001] The present invention relates to an ortho-image creation method for creating an ortho-image based on a captured image captured from above by, for example, an unmanned aircraft, an ortho-image creation system, a three-dimensional model creation method, a three-dimensional model creation system, and a marker used therefor.

Background Art

[0002] Conventionally, when damage such as cracks occurs on the surface of asphalt pavement that constitutes the surface layer of a road, it is necessary to repair the road.

[0003] In order to repair a road, various surveys are conducted, such as an investigation of the road condition (for example, the crack state) at the start time of repair work and the position of planar elements including section lines such as the ends of the road and lane marking lines. Conventionally, for example, the investigation of the crack state has been carried out by visual inspection by an inspector to check the locations where cracks have occurred on the road and the amount of cracks. Instead of the inspector detecting cracks, in some cases, a dedicated road surface property measurement vehicle has been used to conduct an investigation of the road condition (see Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The work of inspecting a road by an inspector to detect cracks is very complicated. When conducting an investigation of the road condition using a dedicated road surface property measurement vehicle, it is necessary to drive the road surface property measurement vehicle, but for a narrow road, it is impossible for the road surface property measurement vehicle to drive, and it is impossible to conduct an investigation of the road condition.

[0006] <0To address the aforementioned technical challenges, one possible approach is to photograph the road using an unmanned aerial vehicle (UAV) flying overhead during the repair work commencement, create an orthomosaic image of the road at the time of the repair work commencement based on the captured images, and then detect cracks on the road surface from the orthomosaic image.

[0007] However, if there are obstacles, such as street trees surrounding a road that cover the widthwise edges of the road surface, it is impossible to photograph the portion of the road surface covered by the obstacles even when photographing the road surface with an unmanned aerial vehicle (UAV) flying overhead. Therefore, it is impossible to create an orthomosaic image that includes the entire road surface, and it is impossible to investigate cracks in the road surface covered by obstacles. Furthermore, if obstacles cover the widthwise edges of the road surface, it is impossible to photograph the portion of the road surface covered by the obstacles even when photographing the road surface with an UAV flying overhead, making it impossible to investigate the width of the road (the position of the road edges) and the position of planar elements including lane markings. In addition, when photographing a road with an UAV flying overhead, other obstacles similar to those mentioned above include pedestrian bridges and traffic lights located above the road surface.

[0008] The present invention has been made in view of these problems, and aims to provide an orthomosaic image creation method, an orthomosaic image creation system and signs used therein, a 3D model creation method, a 3D model creation system and signs used therein, which enable easy investigation of road conditions at the time of repair work commencement based on images taken from above, even when there are obstacles covering a part of the road surface. [Means for solving the problem]

[0009] To solve these problems, the present invention employs the following means.

[0010] In other words, the orthomosaic image creation method according to the present invention includes: a first imaging step of obtaining a plurality of first images by imaging a road in which a portion of the road surface is covered by an obstacle when viewed from above, from an altitude higher than the obstacle using a first imaging device; a second imaging step of obtaining a plurality of second images by imaging the area covered by the obstacle from an altitude lower than the obstacle using a second imaging device; a first coordinate acquisition step of obtaining the three-dimensional coordinates of a first feature point that is located outside the area covered by the obstacle and is included in at least two of the plurality of first images; and at least two of the area covered by the obstacle The orthophoto creation step is characterized by comprising: a second coordinate acquisition step of acquiring the three-dimensional coordinates of second feature points included in the plurality of second captured images; and an orthophoto creation step of creating a corrected orthophoto in which at least a portion of the area of ​​the road surface covered by the obstacle is corrected to an area not covered by the obstacle, based on the plurality of first captured images taken by the first capture step, the plurality of second captured images taken by the second capture step, the three-dimensional coordinates of the first feature points acquired by the first coordinate acquisition step, and the three-dimensional coordinates of the second feature points acquired by the second coordinate acquisition step.

[0011] The orthomosaic image creation system according to the present invention includes: a first image storage means for storing a plurality of first images taken by a first imaging device from an altitude higher than the obstacle, of a road in which a portion of the road surface is covered by an obstacle when viewed from above; a second image storage means for storing a plurality of second images taken by a second imaging device from an altitude lower than the obstacle, of the area covered by the obstacle; a first coordinate storage means for storing the three-dimensional coordinates of a first feature point located outside the area covered by the obstacle and included in at least two of the plurality of first images; and located in the area covered by the obstacle and The system is characterized by comprising: a second coordinate storage means for storing the three-dimensional coordinates of second feature points included in at least two of the plurality of second captured images; and an orthoimage creation means for creating a corrected orthoimage in which at least a portion of the area of ​​the road surface covered by the obstacle is corrected to an area not covered by the obstacle, based on the plurality of first captured images stored in the first captured image storage means, the plurality of second captured images stored in the second captured image storage means, the three-dimensional coordinates of the first feature points stored in the first coordinate storage means, and the three-dimensional coordinates of the second feature points stored in the second storage means.

[0012] As a result, the orthomosaic image creation method and orthomosaic image creation system according to the present invention make it possible to create a corrected orthomosaic image in which the area of ​​the road surface covered by an obstacle is corrected to the area not covered by the obstacle, even when a portion of the road surface is covered by an obstacle when viewed from above. Therefore, even if there is an obstacle covering a portion of the road surface, it is possible to easily investigate the road condition at the time of repair work based on the image taken from above.

[0013] The orthoimage creation method according to the present invention is characterized in that the orthoimage creation step comprises: a first orthoimage creation step of creating a provisional orthoimage in which a part of the road surface is covered by the obstacle, based on a plurality of first images captured by the first shooting step and the three-dimensional coordinates of the first feature points acquired by the first coordinate acquisition step; a shape detection step of detecting the shape of at least a part of the road surface and a part other than the road surface in the area covered by the obstacle, based on a plurality of second images captured by the second shooting step and the three-dimensional coordinates of the second feature points acquired by the second landmark acquisition step; and a second orthoimage creation step of creating a corrected orthoimage in which at least a part of the area covered by the obstacle in the provisional orthoimage created by the first orthoimage creation step is corrected to an area not covered by the obstacle.

[0014] In the orthoimage creation system according to the present invention, the orthoimage creation means comprises: a first orthoimage creation means that creates a provisional orthoimage in which a part of the road surface is covered by an obstacle, based on a plurality of first captured images stored in the first image storage means and the three-dimensional coordinates of the first feature points stored in the first coordinate storage means; a shape detection means that detects the shape of at least a part of the road surface and a part other than the road surface in the area covered by the obstacle, based on a plurality of second captured images stored in the second capture storage means and the three-dimensional coordinates of the second feature points stored in the second coordinate storage means; and a second orthoimage creation means that creates a corrected orthoimage in which at least a part of the area covered by the obstacle in the provisional orthoimage created by the first orthoimage creation means is corrected to an area not covered by the obstacle.

[0015] As a result, the orthomosaic image creation method and orthomosaic image creation system according to the present invention make it possible to create a corrected orthomosaic image in which the area of ​​the road surface covered by the obstacle is corrected to an area not covered by the obstacle in the provisional orthomosaic image, even when a portion of the road surface is covered by an obstacle when viewed from above. Therefore, even if there is an obstacle covering a portion of the road surface, it is possible to easily investigate the road condition at the time of repair work based on the image taken from above.

[0016] In the orthomosaic image creation method according to the present invention, the first imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude higher than the obstacle, and the second imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude lower than the obstacle, or a camera positioned at an altitude lower than the obstacle.

[0017] In the orthomosaic image creation system according to the present invention, the first imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude higher than the obstacle, and the second imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude lower than the obstacle, or a camera positioned at an altitude lower than the obstacle.

[0018] As a result, the orthomosaic image creation method and orthomosaic image creation system according to the present invention make it possible to easily obtain a plurality of first images taken from an altitude higher than the obstacles on a road surface not covered by obstacles, and a plurality of second images taken from an altitude lower than the obstacles on a road surface covered by obstacles, using an unmanned aerial vehicle or model aircraft.

[0019] In the orthomosaic image creation method according to the present invention, the second shooting step is characterized in that a plurality of signs are placed on the road surface and the obstacle in the area covered by the obstacle, and the images are taken such that the plurality of signs are included as second feature points in at least two of the second images.

[0020] In the orthomosaic image creation system according to the present invention, the plurality of second images stored in the second image storage means are characterized in that they are captured such that the road surface and the plurality of signs installed on the obstacle in the area covered by the obstacle are included as second feature points in at least two of the second images.

[0021] As a result, the orthomosaic image creation method and orthomosaic image creation system according to the present invention make it possible to accurately grasp the shape of the road surface and the obstacle in the area covered by the obstacle using multiple second images taken from an altitude lower than the obstacle.

[0022] The sign according to the present invention is a sign used in the orthomosaic image creation method according to the present invention, and is characterized by being a sticker-like sign with an adhesive layer formed on its back surface. As a result, the sign according to the present invention can be easily fixed to the installation location.

[0023] The sign according to the present invention is a sign used in the orthomosaic image creation system according to the present invention, and is characterized by being a sticker-like sign with an adhesive layer formed on its back surface. As a result, the sign according to the present invention can be easily fixed to the installation location.

[0024] The 3D model creation method according to the present invention includes: a first imaging step of imaging a road with a part of the road surface covered by an obstacle from a height higher than the obstacle using a first imaging device to obtain a plurality of first imaging images when viewed from above; a second imaging step of imaging the area covered by the obstacle from a height lower than the obstacle using a second imaging device to obtain a plurality of second imaging images; a first coordinate acquisition step of acquiring 3D coordinates of first feature points that are outside the area covered by the obstacle and are included in at least two of the plurality of first imaging images; a second coordinate acquisition step of acquiring 3D coordinates of second feature points that are in the area covered by the obstacle and are included in at least two of the plurality of second imaging images; and a 3D model creation step of creating a corrected 3D model in which at least a part of the area of the road surface covered by the obstacle is corrected to an area not covered by the obstacle based on the plurality of first imaging images captured in the first imaging step, the plurality of second imaging images captured in the second imaging step, the 3D coordinates of the first feature points acquired in the first coordinate acquisition step, and the 3D coordinates of the second feature points acquired in the second coordinate acquisition step.

[0025] The 3D model creation system according to the present invention includes: a first image storage means for storing a plurality of first images taken by a first imaging device from an altitude higher than the obstacle, of a road in which a portion of the road surface is covered by an obstacle when viewed from above; a second image storage means for storing a plurality of second images taken by a second imaging device from an altitude lower than the obstacle, of the area covered by the obstacle; a first coordinate storage means for storing the 3D coordinates of a first feature point located outside the area covered by the obstacle and included in at least two of the plurality of first images; and located in the area covered by the obstacle and The invention is characterized by comprising: a second coordinate storage means for storing the three-dimensional coordinates of second feature points included in at least two of the plurality of second captured images; and a three-dimensional model creation means for creating a corrected three-dimensional model in which at least a portion of the area of ​​the road surface covered by the obstacle is corrected to an area not covered by the obstacle, based on the plurality of first captured images stored in the first captured image storage means, the plurality of second captured images stored in the second captured image storage means, the three-dimensional coordinates of the first feature points stored in the first coordinate storage means, and the three-dimensional coordinates of the second feature points stored in the second coordinate storage means.

[0026] As a result, the 3D model creation method and 3D model creation system according to the present invention make it possible to create a corrected 3D model in which the area of ​​the road surface covered by an obstacle is corrected to the area not covered by an obstacle, even when a portion of the road surface is covered by an obstacle when viewed from above. Therefore, even if there is an obstacle covering a portion of the road surface, it is possible to investigate the width of the road (position of the road edge) and the position of planar elements including lane markings, etc., at the time of repair work based on images taken from above.

[0027] In the method for creating a three-dimensional model according to the present invention, the three-dimensional model creation step includes: a first creation step of creating a temporary three-dimensional model in which a part of the road surface is covered by the obstacle based on a plurality of first captured images captured in the first capturing step and the three-dimensional coordinates of the first feature points acquired in the first coordinate acquisition step; a shape detection step of detecting at least a part of the shape of the road surface and a part other than the road surface in the area covered by the obstacle based on a plurality of second captured images captured in the second capturing step and the three-dimensional coordinates of the second feature points acquired in the second coordinate acquisition step; and a second three-dimensional model creation step of creating a corrected three-dimensional model in which at least a part of the area covered by the obstacle in the temporary three-dimensional model created in the first creation step is corrected to an area not covered by the obstacle.

[0028] In the three-dimensional model creation system according to the present invention, the three-dimensional model creation means includes: a first three-dimensional model creation means for creating a temporary three-dimensional model in which a part of the road surface is covered by the obstacle based on a plurality of first captured images stored in the first captured image storage means and the three-dimensional coordinates of the first feature points stored in the first coordinate storage means; a shape detection means for detecting at least a part of the shape of the road surface and a part other than the road surface in the area covered by the obstacle based on a plurality of second captured images stored in the second captured image storage means and the three-dimensional coordinates of the second feature points stored in the second coordinate storage means; and a second three-dimensional model creation means for creating a corrected three-dimensional model in which at least a part of the area covered by the obstacle in the temporary three-dimensional model created by the first three-dimensional model creation means is corrected to an area not covered by the obstacle.

[0029] As a result, the 3D model creation method and 3D model creation system according to the present invention make it possible to create a corrected 3D model in which the area of ​​the road surface covered by the obstacle is corrected to an area not covered by the obstacle in the provisional 3D model, even when a portion of the road surface is covered by an obstacle when viewed from above. Therefore, even if there is an obstacle covering a portion of the road surface, it is possible to investigate the width of the road (position of the road edge) and the position of planar elements including lane markings, etc., at the time of repair work based on images taken from above.

[0030] In the method for creating a three-dimensional model according to the present invention, the first imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude higher than the obstacle, and the second imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude lower than the obstacle, or a camera positioned at an altitude lower than the obstacle.

[0031] In the three-dimensional model creation system according to the present invention, the first imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude higher than the obstacle, and the second imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude lower than the obstacle, or a camera positioned at an altitude lower than the obstacle.

[0032] As a result, the 3D model creation method and 3D model creation system according to the present invention make it possible to easily obtain a plurality of first images taken from an altitude higher than the obstacles of a road surface that is not covered by obstacles, and a plurality of second images taken from an altitude lower than the obstacles of a road surface that is covered by obstacles, using an unmanned aerial vehicle or model aircraft.

[0033] In the method for creating a three-dimensional model according to the present invention, the second shooting step is characterized in that a plurality of signs are placed on the road surface and the obstacle in the area covered by the obstacle, and the images are taken such that the plurality of signs are included as second feature points in at least two of the second captured images.

[0034] In the three-dimensional model creation system according to the present invention, the plurality of second images stored in the second image storage means are characterized in that they are captured such that the road surface and a plurality of signs installed on the obstacle in the area covered by the obstacle are included as second feature points in at least two of the second images.

[0035] As a result, the 3D model creation method and 3D model creation system according to the present invention make it possible to accurately grasp the shape of the road surface and the obstacle in the area covered by the obstacle using multiple second images taken from an altitude lower than the obstacle.

[0036] The sign according to the present invention is a sign used in the three-dimensional model creation method according to the present invention, and is characterized by being a sticker-like sign with an adhesive layer formed on its back surface. As a result, the sign according to the present invention can be easily fixed to the installation location.

[0037] The sign according to the present invention is a sign used in the 3D model creation system according to the present invention, and is characterized by being a sticker-like sign with an adhesive layer formed on its back surface. As a result, the sign according to the present invention can be easily fixed to the installation location. [Effects of the Invention]

[0038] As described above, according to the present invention, even when there are obstacles covering a portion of the road surface, it is possible to easily investigate the road condition at the time of repair work based on images taken from above. [Brief explanation of the drawing]

[0039] [Figure 1] This figure shows a schematic configuration of an orthomosaic image creation system according to an embodiment of the present invention. [Figure 2] This diagram shows a road photographed from above, with multiple aerial markers installed near both ends of the road. [Figure 3] This is a diagram showing an aerial marker. [Figure 4] This diagram illustrates a situation where an aerial marker is included in two captured images. [Figure 5] This diagram illustrates the method by which orthomosaic images are created in an orthomosaic imaging device. [Figure 6] This diagram shows a UAV flying at an altitude higher than an obstacle. [Figure 7] This figure shows a hypothetical orthomosaic image created based on images taken by a UAV flying at an altitude higher than the obstacle. [Figure 8] This diagram shows the situation where an aerial marker is installed in an area covered by obstacles. [Figure 9] This diagram shows a UAV flying at an altitude lower than an obstacle. [Figure 10] This figure shows a corrected orthomosaic image created by removing obstacles from a provisional orthomosaic image. [Figure 11] This diagram illustrates the difference in the areas that are rendered in 3D between a provisional orthomosaic image and a corrected orthomosaic image. [Figure 12] This is a magnified view of a road surface where cracks have formed. [Figure 13] This is a magnified view of a road surface where cracks have formed. [Figure 14] This figure shows a schematic configuration of a 3D model creation system according to an embodiment of the present invention. [Figure 15] This diagram illustrates the method by which a 3D model is created using a 3D model creation device. [Figure 16] This is a 3D model of the road surface as seen when photographed from above. [Figure 17] This is a 3D model of the road surface as seen when photographed from above. [Figure 18] This is a 3D model of the road surface showing the state after obstacles have been removed from the 3D model in Figure 16. [Figure 19] This is a 3D model of the road surface, showing the state after obstacles have been removed from the 3D model in Figure 17. [Figure 20]This is a 3D model of the road surface as seen when photographed from above. [Figure 21] This is a 3D model of the road surface, showing the state after obstacles have been removed from the 3D model in Figure 20. [Figure 22] This figure shows a schematic configuration of an orthomosaic image creation system according to a modified example of the present invention. [Figure 23] This diagram illustrates the operation of taking pictures with a camera positioned at an altitude lower than any obstacle. [Modes for carrying out the invention]

[0040] Embodiments of the present invention will be described below with reference to the drawings. In this embodiment, a case will be described in which there are street trees that act as obstacles around the road, and when the area around the road is photographed from above, a part of the road surface is covered by these street trees.

[0041] An orthomosaic image creation system 1 according to an embodiment of the present invention comprises a total station 2 installed at a known point (e.g., a reference point), an unmanned aerial vehicle (UAV3) used as an imaging device, and an orthomosaic image creation device 10 to which the total station 2 and UAV3 are wirelessly connected.

[0042] The total station 2 emits distance measuring light toward each point on the road surface, receives the reflected light reflected at each point, and obtains the 3D coordinates of each point relative to known points based on the number of times the light wave oscillated from emission to reception, and supplies these 3D coordinates to the orthomosaic image creation system 10. In this embodiment, the total station 2 is used to obtain the 3D coordinates of a plurality of ground control markers 6.

[0043] The UAV3 is equipped with an imaging device that photographs the road surface from above, acquires the image data, and supplies the image data to the orthomosaic image creation device 10. The UAV3 can take images while flying at an altitude higher than obstacles, as well as while flying at an altitude lower than obstacles.

[0044] The orthomosaic image creation device 10 is composed of, for example, a microcomputer, and includes a CPU, a ROM containing a program that controls the operation of the orthomosaic image creation device 10, and a RAM that temporarily stores data used when executing the program.

[0045] As shown in Figure 1, the orthomosaic image creation device 10 includes a coordinate storage unit 11, an image capture storage unit 12, an orthomosaic image creation unit 13, a shape detection unit 14, and a display control unit 15. The coordinate storage unit 11 includes a first coordinate storage unit 11a and a second coordinate storage unit 11b. The image capture storage unit 12 includes a first image capture storage unit 12a and a second image capture storage unit 12b. The orthomosaic image creation unit 13 includes a first orthomosaic image creation unit 13a and a second orthomosaic image creation unit 13b. The orthomosaic image creation device 10 also includes a display unit 5, such as a display screen.

[0046] The coordinate storage unit 11 stores the three-dimensional coordinates of multiple feature points, such as ground control markers 6, acquired separately by the total station 2. The first coordinate storage unit 11a stores the three-dimensional coordinates of ground control markers 6 (first feature points) installed outside the area covered by the obstacle when the UAV 3 is flying at an altitude higher than the obstacle covering part of the road surface. The second coordinate storage unit 11b stores the three-dimensional coordinates of ground control markers 6 (second feature points) installed in the area covered by the obstacle when the UAV 3 is flying at an altitude lower than the obstacle covering part of the road surface.

[0047] The captured image storage unit 12 stores multiple images of the road taken from above by the UAV3 flying at a nearly constant altitude above the road. The first captured image storage unit 12a stores multiple first captured images of the road, where a portion of the road surface is covered by an obstacle when viewed from above, taken from an altitude higher than the obstacle. The second captured image storage unit 12b stores multiple second captured images of the area covered by the obstacle, taken from an altitude lower than the obstacle.

[0048] In this embodiment, the UAV3 is flying at an altitude of 20 meters or less above the ground at the time of shooting, for example, at an altitude of 3 to 20 meters, preferably at an altitude of 3 to 15 meters.

[0049] Obstacles around roads that cover part of the road surface when viewed from above are, for example, about 3 to 10 meters high. Therefore, when the UAV3 is taking pictures at an altitude higher than the obstacles, it flies at an altitude of, for example, 10 to 20 meters above the ground, and when it is taking pictures at an altitude lower than the obstacles, it flies at an altitude of, for example, 3 to 10 meters above the ground.

[0050] When photographing a road with a UAV3 flying at an altitude higher than any obstacles, as shown in Figure 2, multiple ground control markers 6 are placed as feature points near both ends of the road. The multiple ground control markers 6 are placed in areas not covered by obstacles and are placed at intervals of, for example, 5 to 15 meters along the ends of the road (in the longitudinal direction of the road). The multiple ground control markers 6 are placed with consideration for creating an orthomosaic image by connecting multiple images taken from above. The ground control markers 6 are feature points for which 3D coordinates are supplied and are used as evaluation points. When creating an orthomosaic image by connecting multiple images, feature points included in the multiple images for which 3D coordinates are not supplied may be used in addition to the ground control markers 6.

[0051] As shown in Figure 3, the aerial marker 6 is a square-shaped plate-like member. The aerial marker 6 has a pattern that clearly indicates its center position. The aerial marker 6 has an adhesive layer formed on its back surface, and a backing paper is attached to cover the adhesive layer, making it a sticker-like material. By removing the backing paper and sticking it to the road, it can be easily fixed to the installation location. Therefore, when using the aerial marker 6, the backing paper covering the adhesive layer is removed, and the back surface of the aerial marker 6 is attached to the road surface. In this embodiment, the aerial marker 6 is, for example, a 9cm x 9cm square, but the type, shape, size, pattern, etc. of the aerial marker 6 are not limited to this.

[0052] As shown in Figure 4, multiple images taken by the UAV3 flying at an altitude higher than the obstacle are captured such that each ground control marker 6 is included in at least two of the images. Therefore, at least one common ground control marker 6 is captured in two adjacent images. Although Figure 4 illustrates the case where all images include ground control marker 6, the multiple images taken by the UAV3 may be captured such that at least two of the images include either ground control marker 6 or other feature points.

[0053] When the ground control marker 6 is photographed by a UAV3 flying at an altitude higher than the obstacle covering part of the road surface, it is placed outside the area covered by the obstacle, as shown in Figure 2. In contrast, when the ground control marker 6 is photographed by a UAV3 flying at an altitude lower than the obstacle covering part of the road surface, it is placed on the road surface or part of the obstacle within the area covered by the obstacle, as shown in Figure 8.

[0054] The orthomosaic image creation unit 13 creates an orthomosaic image based on the 3D coordinates of the ground control marker 6 stored in the coordinate storage unit 11 and multiple captured images stored in the captured image storage unit 12. Specifically, the orthomosaic image creation unit 13 performs SfM (Structure from Motion) analysis on the data of the multiple captured images to connect two adjacent captured images based on the common ground control marker 6 captured in them, create a 3D model based on 3D data (point cloud data), and then creates an orthomosaic image based on that 3D model.

[0055] In more detail, the first orthomosaic image creation unit 13a creates a provisional orthomosaic image based on multiple first captured images stored in the first image storage unit 12a and the three-dimensional coordinates of the aerial markers 6 installed outside the area covered by obstacles, which are stored in the first coordinate storage unit 11a. In the provisional orthomosaic image, a portion of the road surface is covered by obstacles.

[0056] The second orthoimage creation unit 13b creates a corrected orthoimage based on the temporary orthoimage created by the first orthoimage creation unit 13a and the shape (3D shape) of the road surface and parts other than the road surface (including the obstacle) in the area covered by the obstacle detected by the shape detection unit 14. Specifically, the second orthoimage creation unit 13b specifies a predetermined range in the temporary orthoimage that includes the area covered by the obstacle, and replaces that predetermined range with the shape (mesh data) detected by the shape detection unit 14, thereby creating a corrected orthoimage in which the area covered by the obstacle is corrected to an area not covered by the obstacle. In the method of creating a corrected orthoimage of this embodiment, the temporary orthoimage is used as the background, a correction orthoimage (mesh data → orthoimage) is created on that background, and the correction orthoimage is integrated with the temporary orthoimage to create a corrected orthoimage (ultimately, it is the orthoimages that are integrated with each other). Therefore, the integration method for creating a corrected orthoimage of this embodiment corresponds to the integration method for creating a corrected orthoimage described later (integration method 3 for creating a corrected orthoimage).

[0057] The shape detection unit 14 creates mesh data corresponding to the shape (3D shape) of the road surface and parts other than the road surface (including the obstacles) in the area covered by the obstacles, based on a plurality of second captured images stored in the second image storage unit 12b and the 3D coordinates of the aerial markers 6 installed in the area covered by the obstacles, stored in the second coordinate storage unit 11b.

[0058] The display control unit 15 displays the orthophoto image created by the orthophoto creation unit 13 on the display unit 5.

[0059] (Creation of orthomosaic images) The method for creating orthomosaic images using the orthomosaic image creation device 10 will be explained with reference to Figure 5. In this embodiment, when there are obstacles around the road and a portion of the road surface is covered by the obstacles when viewed from above, a method for creating an orthomosaic image of the road surface that is not covered by the obstacles will be described.

[0060] In step S1 (first coordinate acquisition step), the total station 2 acquires three-dimensional coordinates, i.e., planar position (latitude, longitude) and elevation (height), for multiple predetermined locations, i.e., multiple aerial markers 6, around the repair site where road repairs are to be carried out.

[0061] In step S2 (first shooting step), as shown in Figures 6(a) and 6(b), the road is photographed from above by a UAV3 flying at an altitude higher than the obstacles T around the road. Figure 6(b) shows an example of the flight path of the UAV3. At the time of shooting, multiple ground control markers 6 have been pre-installed at multiple predetermined locations surveyed in step S1. Therefore, multiple images are taken such that each ground control marker 6 is included in at least two of the images.

[0062] In step S3 (first orthomosaic image creation step), a provisional orthomosaic image is created based on the 3D coordinates obtained in step S1 and the multiple images taken in step S2. As shown in Figure 7, the provisional orthomosaic image shows that a portion of the road surface is covered by an obstacle T.

[0063] In step S4, in the area where a portion of the road surface is covered by the obstacle T, multiple aerial markers 6 are installed on the road surface and on the sides of the obstacle T, as shown in Figure 8. The area covered by the obstacle T is the area that is hidden by the obstacle T when viewed from above. In Figure 8, the aerial markers 6 are installed on the road-facing side of the obstacle T.

[0064] In step S5 (second coordinate acquisition step), the total station 2 acquires three-dimensional coordinates, i.e., planar position (latitude, longitude) and elevation (height), for predetermined locations where multiple aerial markers 6 are installed in the area covered by the obstacle T.

[0065] In step S6 (second imaging step), as shown in Figures 9(a) and 9(b), the road surface and obstacles T are photographed by a UAV3 flying at an altitude lower than the obstacles T around the road. Figure 9(b) shows an example of the trajectory of the UAV3. Therefore, for each of the multiple ground control markers 6, multiple images are taken such that each ground control marker 6 is included in at least two images.

[0066] In step S7 (shape detection step), mesh data corresponding to the shape of the road surface and parts other than the road surface (including the obstacle T) in the area covered by the obstacle T is created based on a plurality of second captured images stored in the second image storage unit 12b and the three-dimensional coordinates of the aerial marker 6 installed in the area covered by the obstacle T stored in the coordinate storage unit 11b.

[0067] In step S8 (second orthomosaic image creation step), a corrected orthomosaic image is created, as shown in Figure 10, in which the road surface is not covered by obstacle T, based on the provisional orthomosaic image created in step S3 and the mesh data created in step S7 that corresponds to the shape of the road surface and the parts other than the road surface (including obstacle T) in the area covered by obstacle T. In the provisional orthomosaic image in Figure 7, a part of the road surface is covered by obstacle T, and the road surface in that area is hidden, whereas in the corrected orthomosaic image in Figure 10, the obstacle T around the road is cut off near its lower edge, and the road surface that was hidden in Figure 7 is visible.

[0068] In step S9 (display step), a corrected orthomosaic image showing the road surface not covered by obstacles T is displayed on the display unit 5. In this embodiment, the ground pixel dimensions of the orthomosaic image are 5 millimeters or less.

[0069] The difference in the areas that are rendered in 3D between the provisional orthomosaic image (Figure 7) and the corrected orthomosaic image (Figure 10) will be explained based on Figure 11.

[0070] For example, in a provisional orthomosaic image created based on images taken by a UAV3 flying at an altitude higher than the obstacle T around the road, there are no images of the area covered by the obstacle T. As a result, as shown in Figure 11(a), the area below the obstacle T is not formed. Even if the obstacle T were cropped near its lower edge in Figure 11(a), as shown in Figure 11(b), there is no information about the edge of the road surface (the edge on the side closer to the obstacle T), making it impossible to fill in the edge of the road surface.

[0071] In contrast, when using the shape (mesh data) of the road surface and the parts other than the road surface (including the obstacle T) in the area covered by the obstacle T, the area below the obstacle T is also rendered in 3D, as shown in Figure 11(c). Not all of the area below the obstacle T is rendered in 3D, but at least the part near the ground is rendered in 3D. Therefore, if the obstacle T is cut out near its lower end in Figure 11(c), the area not covered by the obstacle T can be corrected by supplementing the edge of the road surface (the edge on the side closer to the obstacle T), as shown in Figure 11(d).

[0072] (Road survey method using orthomosaic images) As described above, the orthomosaic images created by the orthomosaic image creation device 10 are used for various surveys conducted when road repairs are carried out.

[0073] For example, using orthomosaic images created by the orthomosaic image creation device 10, the following can be performed: (1) an investigation of the crack condition of the road surface, (2) an investigation of the location of planar elements around the road including the area to be repaired, (3) an investigation for repairing the area around manholes, (4) an investigation of the distance between two designated points on the road surface, and (5) an investigation of the area of ​​a designated range on the road surface.

[0074] Figures 12 and 13 are enlarged views of a road surface where cracks have formed. As shown above, the orthoimage created by the orthoimage creation device 10 of this embodiment makes it possible to clearly identify, for example, cracks formed on the road surface. Therefore, based on the orthoimage displayed on the display unit 5, an investigation into the cracking state of the road surface is conducted to determine the locations where cracks have formed on the road surface, including the repair areas where road repairs are to be carried out, as well as the degree of cracking and patching in those areas.

[0075] As mentioned above, when a portion of the road surface is covered by an obstacle when viewed from above, an orthomosaic image is created of the road surface that is not covered by the obstacle by removing the obstacle. In the process of creating the orthomosaic image, Structure from Motion (SfM) analysis is performed on the data from multiple captured images, and two adjacent captured images are connected based on common aerial markers 6 captured on them to create a 3D model (a 3D model based on 3D data (point cloud data)).

[0076] (Creation of 3D models) A 3D model creation system and method for creating a 3D model of the area around a road will be described based on Figures 14 and 15. In this embodiment, a method for creating a 3D model of a road where the road surface is not covered by obstacles will be described, when there are obstacles around the road and a portion of the road surface is covered by the obstacles when viewed from above.

[0077] As shown in Figure 14, the 3D model creation system 101 according to an embodiment of the present invention includes a total station 2 installed at a known point (e.g., a reference point), an unmanned aerial vehicle (UAV3) used as an imaging device, and an orthomosaic image creation device 110 to which the total station 2 and UAV3 are wirelessly connected.

[0078] In other words, by changing the following in the orthomosaic image creation system shown in Figure 1, a 3D model creation system is shown in Figure 14, which creates a 3D model using the 3D model creation method of this embodiment. The orthomosaic image creation unit 13 is replaced with a 3D model creation unit 113 that creates a 3D model. The first orthomosaic image creation unit 13a is replaced with the first creation unit 113a which creates a provisional 3D model. The second orthomosaic image creation unit 13b is replaced with a second creation unit 113b that creates a corrected 3D model. Furthermore, by changing the following in the orthomosaic image creation method shown in Figure 5, the 3D model creation method of this embodiment is shown in Figure 15. Step S3: Change the creation of a provisional orthomosaic image to the creation of a provisional 3D model. Step S8: Change the creation of corrected orthomosaic images to the creation of corrected 3D models. Step S9: Change the display from the corrected orthomosaic image to the corrected 3D model.

[0079] As described above, a corrected orthomosaic image in which the road surface is not covered by obstacle T is created based on a provisional orthomosaic image in which part of the road surface is covered by obstacle T and mesh data corresponding to the shape of the road surface and parts other than the road surface (including obstacle T) in the area covered by obstacle T. Similarly, a corrected 3D model in which the road surface is not covered by obstacle T is created based on a provisional 3D model in which part of the road surface is covered by obstacle T and point cloud data corresponding to the shape of the road surface and parts other than the road surface (including obstacle T) in the area covered by obstacle T, so a detailed explanation is omitted. For this reason, in the 3D model creation system 101 shown in Figure 14, the shape detection unit 14 creates point cloud data corresponding to the shape (3D shape) of the road surface and parts other than the road surface (including obstacle) in the area covered by obstacle, based on a plurality of second captured images stored in the second capture memory unit 12b and the 3D coordinates of the aerial marker 6 installed in the area covered by obstacle stored in the second coordinate memory unit 11b.

[0080] In the method for creating a corrected 3D model of this embodiment, a provisional 3D model is used as the background, a corrective 3D model (mesh data) is created on top of that background, and the corrective 3D model is integrated into the provisional 3D model to create the corrected 3D model. Therefore, the integration method for creating the corrected 3D model of this embodiment corresponds to the integration method 1 for creating the corrected 3D model described later.

[0081] Furthermore, by rotating the 3D model of the road surface on the display screen, for example, it is possible to change the 3D model of the road surface to a view from various directions. The following explanation describes the case where the 3D model of the road surface is viewed from various directions.

[0082] Figures 16 and 17 are 3D models showing a state where part of the road surface is covered by obstacles such as street trees, traffic lights, and road signs, as seen when the road surface is photographed from above. Therefore, in Figures 16 and 17, part of the road surface is hidden by obstacles and is not visible.

[0083] In contrast, Figures 18 and 19 are 3D models showing the road surface as it appears when photographed from above, with obstacles such as street trees, traffic lights, and road signs that partially obscure the road surface removed. Therefore, in Figures 18 and 19, parts of the road surface that were hidden and invisible due to obstacles in Figures 16 and 17 are now visible.

[0084] Figure 20 is a 3D model created when the road surface was photographed from a different direction than in Figures 16 and 17. In Figure 20, most of the road surface is hidden by street trees. In contrast, Figure 21 is a 3D model showing the state in Figure 20 with the street trees removed. In Figure 21, most of the road surface that was hidden and invisible in Figure 20 is visible.

[0085] Therefore, since Figures 18, 19, and 21 show the entire road surface, it is possible to investigate the width of the road (the position of the road edges) and the positions of planar elements, including lane markings and other road markings.

[0086] The orthomosaic image creation method of this embodiment includes: a first imaging step of obtaining multiple first images by imaging a road in which a portion of the road surface is covered by an obstacle when viewed from above using a first imaging device (UAV3) from an altitude higher than the obstacle; a second imaging step of obtaining multiple second images by imaging the area covered by the obstacle using a second imaging device (UAV3) from an altitude lower than the obstacle; a first coordinate acquisition step of obtaining the three-dimensional coordinates of a first feature point that is located outside the area covered by the obstacle and is included in at least two of the multiple first images; and in the area covered by the obstacle The system includes a second coordinate acquisition step of acquiring the three-dimensional coordinates of second feature points included in at least two of the second captured images, and an orthophoto creation step of creating a corrected orthophoto in which at least a portion of the area of ​​the road surface covered by obstacles is corrected to an area not covered by obstacles, based on the multiple first captured images taken in the first shooting step, the multiple second captured images taken in the second shooting step, the three-dimensional coordinates of the first feature points acquired in the first coordinate acquisition step, and the three-dimensional coordinates of the second feature points acquired in the second coordinate acquisition step.

[0087] The orthomosaic image creation system 1 of this embodiment includes: a first image storage unit 12a that stores a plurality of first images taken by a first imaging device (UAV3) from an altitude higher than the obstacle, of a road where a portion of the road surface is covered by an obstacle when viewed from above; a second image storage unit 12b that stores a plurality of second images taken by a second imaging device (UAV3) from an altitude lower than the obstacle, of the area covered by the obstacle; and a first coordinate storage unit 11a that stores the three-dimensional coordinates of first feature points that are located outside the area covered by the obstacle and are included in at least two of the plurality of first images, and The system includes a second coordinate storage unit 11b that stores the three-dimensional coordinates of second feature points that are located in an area covered by an obstacle and are included in at least two of the second captured images, and an orthoimage creation unit 13 that creates a corrected orthoimage in which at least a portion of the area covered by the obstacle on the road surface is corrected to an area not covered by the obstacle, based on the multiple first captured images stored in the first captured image storage unit 12a, the multiple second captured images stored in the second captured image storage unit 12b, the three-dimensional coordinates of the first feature points stored in the first coordinate storage unit 11a, and the three-dimensional coordinates of the second feature points stored in the second coordinate storage unit 11b.

[0088] As a result, the orthomosaic image creation method and orthomosaic image creation system 1 of this embodiment make it possible to create a corrected orthomosaic image in which the area of ​​the road surface covered by an obstacle is corrected to the area not covered by the obstacle, even when a portion of the road surface is covered by an obstacle when viewed from above. Therefore, even if there is an obstacle covering a portion of the road surface, it is possible to easily investigate the road condition at the time of repair work based on the image taken from above.

[0089] In the orthomosaic image creation method of this embodiment, the orthomosaic image creation step includes: a first orthomosaic image creation step that creates a provisional orthomosaic image in which a part of the road surface is covered by an obstacle, based on a plurality of first captured images taken in a first shooting step and the three-dimensional coordinates of first feature points acquired in a first coordinate acquisition step; a shape detection step that detects the shape of at least a part of the road surface and parts other than the road surface in the area covered by the obstacle, based on a plurality of second captured images taken in a second shooting step and the three-dimensional coordinates of second feature points acquired in a second landmark acquisition step; and a second orthomosaic image creation step that creates a corrected orthomosaic image in which at least a part of the area covered by the obstacle in the provisional orthomosaic image created in the first orthomosaic image creation step is corrected to an area not covered by the obstacle.

[0090] In the orthoimage creation system 1 of this embodiment, the orthoimage creation unit 13 includes: a first orthoimage creation unit 13a that creates a provisional orthoimage in which a part of the road surface is covered by an obstacle, based on a plurality of first captured images stored in a first image storage unit 12a and the three-dimensional coordinates of first feature points stored in a first coordinate storage unit 11a; a shape detection unit 14 that detects the shape of at least a part of the road surface and parts other than the road surface in the area covered by the obstacle, based on a plurality of second captured images stored in a second captured image storage unit 12b and the three-dimensional coordinates of second feature points stored in a second coordinate storage unit 11b; and a second orthoimage creation unit 13b that creates a corrected orthoimage in which at least a part of the area covered by the obstacle in the provisional orthoimage created by the first orthoimage creation unit 13a is corrected to an area not covered by the obstacle.

[0091] As a result, the orthomosaic image creation method and orthomosaic image creation system 1 according to the present invention make it possible to create a corrected orthomosaic image in which the area of ​​the road surface covered by the obstacle is corrected to an area not covered by the obstacle in the provisional orthomosaic image, even when a portion of the road surface is covered by an obstacle when viewed from above. Therefore, even if there is an obstacle covering a portion of the road surface, it is possible to easily investigate the road condition at the time of repair work based on the image taken from above.

[0092] In the orthomosaic image creation method of this embodiment, the first imaging device is a UAV3 flying at an altitude higher than the obstacle, and the second imaging device is a UAV3 flying at an altitude lower than the obstacle.

[0093] In the orthomosaic image creation system 1 of this embodiment, the first imaging device is a UAV3 flying at an altitude higher than the obstacle, and the second imaging device is a UAV3 flying at an altitude lower than the obstacle. That is the case.

[0094] As a result, the orthomosaic image creation method and orthomosaic image creation system 1 of this embodiment make it possible to easily obtain multiple first images taken by the UAV 3 of the road surface not covered by obstacles from an altitude higher than the obstacles, and multiple second images taken by the UAV 3 of the road surface covered by obstacles from an altitude lower than the obstacles.

[0095] In the orthomosaic image creation method of this embodiment, in the second shooting step, a plurality of aerial markers 6 are placed on the road surface and on the obstacles in an area covered by obstacles, and the images are taken such that the plurality of aerial markers 6 are included as second feature points in at least two second images.

[0096] In the orthomosaic image creation system 1 of this embodiment, the multiple second images stored in the second image storage unit 12b are captured such that at least two of the second images include the road surface and multiple aerial markers 6 installed on the obstacles in an area covered by obstacles, as second feature points.

[0097] As a result, the orthomosaic image creation method and orthomosaic image creation system 1 of this embodiment make it possible to accurately grasp the shape of the road surface and the obstacle in the area covered by the obstacle using multiple second images taken from an altitude lower than the obstacle.

[0098] The aerial marker 6 of this embodiment is a marker used in the orthomosaic image creation method of this embodiment, and is in the form of a sticker with an adhesive layer formed on its back surface. This makes it possible to easily fix the aerial marker 6 of this embodiment to the installation location.

[0099] The aerial marker 6 of this embodiment is a marker used in the orthomosaic image creation system 1 of this embodiment, and is in the form of a sticker with an adhesive layer formed on its back surface. This makes it possible to easily fix the aerial marker 6 of this embodiment to the installation location.

[0100] The 3D model creation method of this embodiment includes a first imaging step of obtaining multiple first images by imaging a road in which a portion of the road surface is covered by an obstacle when viewed from above using a first imaging device (UAV3) from an altitude higher than the obstacle; a second imaging step of obtaining multiple second images by imaging the area covered by the obstacle using a second imaging device (UAV3) from an altitude lower than the obstacle; a first coordinate acquisition step of obtaining 3D coordinates of first feature points that are located outside the area covered by the obstacle and are included in at least two of the multiple first images; and a third step of obtaining 3D coordinates of first feature points located in the area covered by the obstacle The method further includes a second coordinate acquisition step of acquiring the three-dimensional coordinates of second feature points included in at least two of the second captured images, and a three-dimensional model creation step of creating a corrected three-dimensional model in which at least a portion of the area of ​​the road surface covered by obstacles is corrected to an area not covered by obstacles, based on the multiple first captured images taken in the first capture step, the multiple second captured images taken in the second capture step, the three-dimensional coordinates of the first feature points acquired in the first coordinate acquisition step, and the three-dimensional coordinates of the second feature points acquired in the second coordinate acquisition step.

[0101] The 3D model creation system 101 of this embodiment includes a first image storage unit 12a that stores a plurality of first images taken by a first imaging device (UAV3) from an altitude higher than the obstacle, of a road in which a portion of the road surface is covered by an obstacle when viewed from above; a second image storage unit 12b that stores a plurality of second images taken by a second imaging device (UAV3) from an altitude lower than the obstacle, of the area covered by the obstacle; a first coordinate storage unit 11a that stores the 3D coordinates of first feature points that are located outside the area covered by the obstacle and are included in at least two of the plurality of first images; and The system includes a second coordinate storage unit 11b that stores the three-dimensional coordinates of second feature points that are located in an area covered by an obstacle and are included in at least two second captured images, and a three-dimensional model creation unit 113 that creates a corrected three-dimensional model in which at least a portion of the area covered by the obstacle on the road surface is corrected to an area not covered by the obstacle, based on a plurality of first captured images stored in a first captured image storage unit 12a, a plurality of second captured images stored in a second captured image storage unit 12b, the three-dimensional coordinates of the first feature points stored in the first coordinate storage unit 11a, and the three-dimensional coordinates of the second feature points stored in the second coordinate storage unit 11b.

[0102] As a result, the 3D model creation method and 3D model creation system 101 of this embodiment make it possible to create a corrected 3D model in which the area of ​​the road surface covered by an obstacle is corrected to the area not covered by an obstacle, even when a portion of the road surface is covered by an obstacle when viewed from above. Therefore, even if there is an obstacle covering a portion of the road surface, it is possible to investigate the width of the road (position of the road edge) and the position of planar elements including lane markings, etc., at the time of repair work, based on images taken from above.

[0103] The 3D model creation method of this embodiment includes a first creation step of creating a provisional 3D model in which a portion of the road surface is covered by an obstacle, based on a plurality of first captured images taken in a first shooting step and the 3D coordinates of first feature points acquired in a first coordinate acquisition step; a shape detection step of detecting the shape of at least a portion of the road surface and parts other than the road surface in the area covered by the obstacle, based on a plurality of second captured images taken in a second shooting step and the 3D coordinates of second feature points acquired in a second landmark acquisition step; and a second 3D model creation step of creating a corrected 3D model in which at least a portion of the area covered by the obstacle in the provisional 3D model created in the first creation step is corrected to an area not covered by the obstacle.

[0104] The 3D model creation system 101 of this embodiment includes a 3D model creation unit 113 which comprises: a first creation unit 113a that creates a provisional 3D model in which a part of the road surface is covered by an obstacle, based on a plurality of first captured images stored in a first captured image storage unit 12a and the 3D coordinates of first feature points stored in a first coordinate storage unit 11a; a shape detection unit 14 that detects the shape of at least a part of the road surface and parts other than the road surface in the area covered by the obstacle, based on a plurality of second captured images stored in a second captured image storage unit 12b and the 3D coordinates of second feature points stored in a second coordinate storage unit 11b; and a second creation unit 113b that creates a corrected 3D model in which at least a part of the area covered by the obstacle in the provisional 3D model created by the first creation unit 113a is corrected to an area not covered by the obstacle.

[0105] As a result, the 3D model creation method and 3D model creation system 101 according to the present invention make it possible to create a corrected 3D model in which the area of ​​the road surface covered by the obstacle is corrected to an area not covered by the obstacle in the provisional 3D model, even when a part of the road surface is covered by an obstacle when viewed from above. Therefore, even if there is an obstacle covering a part of the road surface, it is possible to investigate the width of the road (position of the road edge) and the position of planar elements including lane markings, etc., at the time of repair work based on images taken from above.

[0106] In the three-dimensional model creation method of this embodiment, the first imaging device is a UAV3 flying at an altitude higher than the obstacle, and the second imaging device is a UAV3 flying at an altitude lower than the obstacle.

[0107] In the 3D model creation system 101 of this embodiment, the first imaging device is a UAV3 flying at an altitude higher than the obstacle, and the second imaging device is a UAV3 flying at an altitude lower than the obstacle.

[0108] As a result, the 3D model creation method and 3D model creation system 101 of this embodiment make it possible to easily obtain multiple first images taken by the UAV3 of the road surface not covered by obstacles from an altitude higher than the obstacles, and multiple second images taken by the UAV3 of the road surface covered by obstacles from an altitude lower than the obstacles.

[0109] In the 3D model creation method of this embodiment, in the second shooting step, multiple aerial markers 6 are placed on the road surface and on the obstacles in an area covered by obstacles, and the multiple aerial markers 6 are photographed so that they are included as second feature points in at least two second images.

[0110] In the 3D model creation system 101 of this embodiment, the multiple second images stored in the second image storage unit 12b are captured such that at least two of the second images include the road surface and multiple aerial markers 6 installed on the obstacles in an area covered by obstacles, as second feature points.

[0111] As a result, the 3D model creation method and 3D model creation system 101 of this embodiment make it possible to accurately grasp the shape of the road surface and the obstacle in the area covered by the obstacle using multiple second images taken from an altitude lower than the obstacle.

[0112] The aerial marker 6 of this embodiment is a marker used in the 3D model creation method of this embodiment, and is in the form of a sticker with an adhesive layer formed on its back surface. As a result, the aerial marker 6 of this embodiment can be easily fixed to the installation location.

[0113] The aerial marker 6 of this embodiment is a marker used in the 3D model creation system 101 of this embodiment, and is in the form of a sticker with an adhesive layer formed on its back surface. This makes it possible to easily fix the aerial marker 6 of this embodiment to the installation location.

[0114] Although embodiments of the present invention have been described above, the specific configuration of each part is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the present invention.

[0115] In the above embodiment, a 3D model and orthomosaic image are created based on images taken from above a road by a UAV3 flying at a nearly constant altitude of 20 meters or less above the ground. However, the present invention includes a method for creating a 3D model and orthomosaic image based on images taken from above a road by a UAV3 flying at an altitude greater than 20 meters above the ground.

[0116] In the above embodiment, the three-dimensional coordinates of the ground control marker 6 are obtained by the total station 2, but the three-dimensional coordinates of the ground control marker 6 may be obtained by a GNSS (Global Navigation Satellite System), which is a satellite-based positioning system such as GPS. The three-dimensional coordinates of the ground control marker 6 may be obtained by scanning with the 3D scanner 4. The three-dimensional coordinates of ground control marker 6 installed around a road may be obtained by scanning with the 3D scanner 4. In addition, although the three-dimensional coordinates of a predetermined location where multiple ground control markers 6 are installed are obtained by the total station 2, if the three-dimensional coordinates of a predetermined location have already been obtained, those three-dimensional coordinates may be obtained.

[0117] In the above embodiment, the aerial marker 6 is square in shape and has a pattern that clearly identifies the evaluation point and the central position used. However, the shape of the aerial marker 6 and the central position of the aerial marker 6 are not limited to cases where these are characteristic points. The aerial marker 6 has a pattern that identifies positions other than its central position, and these positions other than the central position may be used as evaluation points.

[0118] Furthermore, in the above embodiment, a plate-shaped aerial marker 6 is installed on the road surface, but instead of using a plate-shaped aerial marker 6, a pattern similar to that of the aerial marker 6 may be formed on the road surface using any material such as paint. For example, a pattern identical in shape to the white portion of the aerial marker 6 in Figure 3 may be formed on the asphalt surface of a road by spraying paint of a different color from the asphalt surface. When an aerial marker is formed on the road surface using any material such as paint, the type, shape, size, and pattern of the aerial marker are also arbitrary. The above also applies when, for example, an aerial marker is fixed to an obstacle such as a street tree.

[0119] Furthermore, in the above embodiment, a case was described in which an aerial marker 6 formed in the form of a sticker with backing paper attached to cover the adhesive layer on the back side is used, and the aerial marker 6 is fixed to the installation location by removing the backing paper and attaching it to the installation location. However, the method of fixing the aerial marker 6 to the installation location is arbitrary. For example, an aerial marker with a hole may be fixed to the installation location with a fixing device (for example, an anchor pin). In that case, a mounting hole may be formed in the installation location such as the road surface, and the aerial marker may be fixed to the mounting hole with a fixing device. Alternatively, for example, a rod-shaped marker stake with an aerial marker formed at its tip may be embedded in the installation location and fixed to the installation location. In that case, by embedding the marker stake in the ground surface, the aerial marker formed at the tip of the marker stake will be positioned on the ground surface. The above contents are also similar when fixing an aerial marker to an obstacle such as a street tree.

[0120] In the above embodiment, a method for creating orthomosaic images was described. However, it is possible to take images after obtaining the three-dimensional coordinates of the aerial marker 6, and it is also possible to obtain the three-dimensional coordinates of the aerial marker 6 after taking the images.

[0121] In the above embodiment, when a portion of the road surface is covered by an obstacle when viewed from above, a corrected 3D model and corrected orthoimage are created by correcting the entire area covered by the obstacle to an area not covered by the obstacle. However, a corrected 3D model and corrected orthoimage may be created by correcting only a portion of the area covered by the obstacle (for example, at least a portion of the road surface and the portion other than the road surface) to an area not covered by the obstacle. Therefore, a corrected 3D model and corrected orthoimage may be created in which the road surface in the area covered by the obstacle is corrected to an area not covered by the obstacle, but the portion other than the road surface in the area covered by the obstacle is not corrected to an area not covered by the obstacle.

[0122] In the above embodiments, the first and second imaging devices used were unmanned aerial vehicles (including imaging devices) flying at an altitude higher than the obstacle and unmanned aerial vehicles (including imaging devices) flying at an altitude lower than the obstacle, respectively, but are not limited to these. In the present invention, the types of the first and second imaging devices are arbitrary. For example, as the first and second imaging devices, at least one of the cases of photographing a road from an altitude higher than the obstacle and photographing a road from an altitude lower than the obstacle may be photographed using a model aircraft (including imaging device) flying at an altitude higher than the obstacle or a model aircraft (including imaging device) flying at an altitude lower than the obstacle. In the present invention, an unmanned aerial vehicle is an airplane, rotary-wing aircraft, airship, etc., that cannot carry a person and is capable of flight by remote control or autopilot, such as a drone (multicopter), radio-controlled aircraft, etc. A model aircraft is, for example, a multicopter, radio-controlled aircraft, etc., with a total weight of less than 200 grams, which is the sum of the weight of the aircraft body and the battery. In the above embodiment, when a road is photographed by an unmanned aerial vehicle flying overhead, the obstacles covering a portion of the road surface are not limited to street trees surrounding the road. The present invention is applicable, for example, when obstacles such as pedestrian bridges or traffic lights located above the road surface cover a portion of the road surface.

[0123] In the above embodiment, the first and second imaging devices used were unmanned aerial vehicles (including imaging devices) flying at an altitude higher than the obstacle and unmanned aerial vehicles (including imaging devices) flying at an altitude lower than the obstacle, respectively. However, the embodiment is not limited to these. For example, as the first and second imaging devices, at least one of the cases where the road is photographed from an altitude higher than the obstacle and the road is photographed from an altitude lower than the obstacle may be photographed using a camera positioned at an altitude higher than the obstacle or a camera positioned at an altitude lower than the obstacle.

[0124] For example, an orthomosaic image creation system according to a modified version of the present invention may include, as shown in Figure 22, a total station 2 installed at a known point (e.g., a reference point), an unmanned aerial vehicle (UAV3) as an imaging device, a camera 103 as an imaging device, and an orthomosaic image creation device 10 in which the total station 2, UAV3 and camera 103 are wirelessly connected.

[0125] In this modified example, when photographing a road where part of the road surface is covered by an obstacle when viewed from above, the UAV3 is used for photography from an altitude higher than the obstacle, and when photographing the area covered by the obstacle from an altitude lower than the obstacle, the camera 103 is used for photography. That is, an inspector who is in the area where part of the road surface is covered by the obstacle T operates the camera 103 to photograph the area (road surface and obstacle T) that includes the road surface and multiple aerial markers 6 installed on the sides of the obstacle T, etc., as shown in Figure 23. In this case, the inspector moves and takes multiple images such that each of the multiple aerial markers 6 is included in at least two images.

[0126] In the above embodiment, when creating an orthomosaic image based on the three-dimensional coordinates of the aerial marker 6 and multiple captured images, SfM (Structure from Motion) analysis software is used. The process performed by the SfM analysis software includes: (1) a point cloud generation process that constitutes a collection of multiple point cloud data corresponding to multiple locations; (2) a TIN process (meshing process that meshes the collection of point cloud data formed in the point cloud generation process), which converts the point cloud data into a three-dimensional TIN model (irregular triangular network), which is a collection of triangular planes connected as vertices; and (3) an orthomosaic image creation process that creates an orthomosaic image based on the three-dimensional TIN model converted in the TIN process. The orthomosaic image (provisional orthomosaic image and corrected orthomosaic image) of the present invention is obtained by performing the orthomosaic image creation process using SfM analysis software, and the three-dimensional model (provisional three-dimensional model and corrected three-dimensional model) of the present invention is obtained by performing the point cloud generation process using SfM analysis software, or by performing the TIN process after the point cloud generation process using SfM analysis software. In other words, a 3D model (point cloud data) is created by performing the point cloud generation process using SfM analysis software, and a 3D model (TIN-generated point cloud data) is also created by performing the TIN process after the point cloud generation process using SfM analysis software.

[0127] (How to create corrected orthomosaic images) In this invention, SfM analysis is performed on multiple images taken from an altitude higher than the obstacle and the 3D coordinates of the aerial markers 6 contained therein, and SfM analysis is also performed on multiple images taken from an altitude lower than the obstacle and the 3D coordinates of the aerial markers 6 contained therein. By integrating this data, a corrected orthomosaic image is created.

[0128] One method for creating a corrected orthomosaic image is to integrate SfM analysis performed on multiple images taken from an altitude higher than the obstacle and the 3D coordinates of the air defense marker 6 contained therein, with SfM analysis performed on multiple images taken from an altitude lower than the obstacle and the 3D coordinates of the air defense marker 6 contained therein. In the above embodiment, the case using (integration method 3 for creating a corrected orthomosaic image) described below was explained, but the integration method for creating a corrected orthomosaic image is arbitrary.

[0129] (Integration method 1 for creating corrected orthomosaic images) Multiple images taken from an altitude lower than the obstacle, along with the 3D coordinates of the air defense marker 6 contained within them, are subjected to SfM analysis up to the point cloud generation stage. These are then integrated with multiple images taken from an altitude higher than the obstacle, along with the 3D coordinates of the air defense marker 6 contained within them, which have also undergone SfM analysis up to the point cloud generation stage. Subsequently, a corrected orthomosaic image is created by performing a TIN (Telescopic Injection) process and an orthomosaic image creation process on the integrated data.

[0130] (Integration method 2 for creating corrected orthomosaic images) Multiple images taken from an altitude lower than the obstacle, along with the 3D coordinates of the air defense marker 6 contained therein, are subjected to SfM analysis up to the point cloud generation and TIN process. These are then integrated with multiple images taken from an altitude higher than the obstacle, along with the 3D coordinates of the air defense marker 6 contained therein, which have also undergone SfM analysis up to the point cloud generation and TIN process. Subsequently, an orthomosaic image is created by performing an orthomosaic image creation process on the integrated data.

[0131] (Integration method 3 for creating corrected orthomosaic images) After performing SfM analysis on multiple images taken from an altitude lower than the obstacle and the 3D coordinates of the air defense marker 6 contained therein, including the point cloud generation and TIN process, and then completing the orthomosaic image creation process, the corrected orthomosaic image is created by integrating these results with multiple images taken from an altitude higher than the obstacle and the 3D coordinates of the air defense marker 6 contained therein, including the point cloud generation and TIN process, and then completing the orthomosaic image creation process.

[0132] (Integration method 4 for creating corrected orthomosaic images) A corrected orthomosaic image is created by performing SfM analysis on a combined set of multiple images taken from an altitude lower than the obstacle and the 3D coordinates of the air defense marker 6 contained therein, and multiple images taken from an altitude higher than the obstacle and the 3D coordinates of the air defense marker 6 contained therein, through a point cloud creation process, a TIN process, and an orthomosaic image creation process.

[0133] (Method for creating a corrected 3D model) In this invention, SfM analysis is performed on multiple images taken from an altitude higher than the obstacle and the 3D coordinates of the ground control markers 6 contained therein, and SfM analysis is also performed on multiple images taken from an altitude lower than the obstacle and the 3D coordinates of the ground control markers 6 contained therein. A corrected 3D model is created by integrating this data.

[0134] One method for creating a corrected 3D model is to integrate SfM analysis performed on multiple images taken from an altitude higher than the obstacle and the 3D coordinates of the air defense marker 6 contained therein, with SfM analysis performed on multiple images taken from an altitude lower than the obstacle and the 3D coordinates of the air defense marker 6 contained therein. In the above embodiment, the case using (integration method 1 for creating a corrected 3D model) described below was explained, but the integration method for creating a corrected 3D model is arbitrary.

[0135] (Integration method 1 for creating a corrected 3D model) A corrected 3D model is created by integrating SfM analysis (up to the point cloud generation) performed on multiple images taken from an altitude lower than the obstacle and the 3D coordinates of the air defense marker 6 contained within them, with SfM analysis (up to the point cloud generation) performed on multiple images taken from an altitude higher than the obstacle and the 3D coordinates of the air defense marker 6 contained within them.

[0136] (Integration method 2 for creating a corrected 3D model) Multiple images taken from an altitude lower than the obstacle, along with the 3D coordinates of the air defense marker 6 contained within them, are subjected to SfM analysis up to the point cloud generation stage. These are then integrated with multiple images taken from an altitude higher than the obstacle, along with the 3D coordinates of the air defense marker 6 contained within them, and the SfM analysis is performed up to the point cloud generation stage. Subsequently, a TIN (Telescopic Injection Generator) process is performed on the integrated data to create a corrected 3D model.

[0137] (Integration method 3 for creating a corrected 3D model) Multiple images taken from an altitude lower than the obstacle, along with the 3D coordinates of the air defense marker 6 contained within them, are subjected to SfM analysis up to the point cloud generation and TIN process. These SfM analyses are then integrated with multiple images taken from an altitude higher than the obstacle, along with the 3D coordinates of the air defense marker 6 contained within them, up to the point cloud generation and TIN process, to create a corrected 3D model.

[0138] (Integration method 4 for creating a corrected 3D model) A corrected 3D model is created by performing SfM analysis up to the point cloud generation stage on a combined set of multiple images taken from an altitude lower than the obstacle and the 3D coordinates of the air defense marker 6 contained therein, and multiple images taken from an altitude higher than the obstacle and the 3D coordinates of the air defense marker 6 contained therein.

[0139] (5 integration methods for creating a corrected 3D model) A corrected 3D model is created by performing SfM analysis on a combined data set of multiple images taken from an altitude lower than the obstacle and the 3D coordinates of the air defense marker 6 contained therein, and multiple images taken from an altitude higher than the obstacle and the 3D coordinates of the air defense marker 6 contained therein, after the point cloud generation process, up to the TIN process.

[0140] In the above-described integration method for creating corrected orthomosaic images or integration method for creating corrected 3D models, SfM analysis performed on multiple images taken from an altitude lower than the obstacle and the 3D coordinates of the air defense marker 6 contained therein was integrated with SfM analysis performed on multiple images taken from an altitude higher than the obstacle and the 3D coordinates of the air defense marker 6 contained therein. However, it is also possible to integrate SfM analysis performed on multiple images taken from an altitude higher than the obstacle and the 3D coordinates of the air defense marker 6 contained therein with SfM analysis performed on multiple images taken from an altitude lower than the obstacle and the 3D coordinates of the air defense marker 6 contained therein. [Explanation of Symbols]

[0141] 1. Orthomosaic Image Creation System 2 Total Station 3 UAV (unmanned aerial vehicle) 4. 3D scanner (3D scanning device) 5 Display section 6. Aerial markers 10 Orthomosaic imaging device 11 Coordinate storage unit 11a First coordinate storage unit (first coordinate storage means) 11b Second coordinate storage unit (second coordinate storage means) 12 Image storage unit 12a First captured image storage unit (first captured image storage means) 12b Second image storage unit (second image storage means) 13 Orthomosaic Image Creation Department 13a First orthomosaic image creation unit (first orthomosaic image creation means) 13b Second orthomosaic image creation unit (second orthomosaic image creation means) 14 Shape detection unit (shape detection means) 15 Display Control Unit 101 3D Model Creation System 103 Camera 113 3D Model Creation Department 113a First 3D model creation unit (first 3D model means) 113b Second 3D Model Creation Unit (Second 3D Modeling Means)

Claims

1. A first shooting step involves taking a photograph of a road, where a portion of the road surface is covered by an obstacle when viewed from above, using a first shooting device from an altitude higher than the obstacle, to obtain a plurality of first images. A second imaging step involves photographing the area covered by the aforementioned obstacle from a height lower than the obstacle using a second imaging device to obtain a plurality of second images, A first coordinate acquisition step of acquiring three-dimensional coordinates of a first feature point that is located outside the area covered by the aforementioned obstacle and is included in at least two of the plurality of first captured images, A second coordinate acquisition step of acquiring three-dimensional coordinates of a second feature point that is located in the area covered by the aforementioned obstacle and is included in at least two of the plurality of second captured images, An orthoimage creation method characterized by comprising: an orthoimage creation step, which creates a corrected orthoimage in which at least a portion of the area of ​​the road surface covered by the obstacle is corrected to an area not covered by the obstacle, based on a plurality of first images captured in the first shooting step, a plurality of second images captured in the second shooting step, the three-dimensional coordinates of the first feature points acquired in the first coordinate acquisition step, and the three-dimensional coordinates of the second feature points acquired in the second coordinate acquisition step.

2. The orthomosaic image creation step described above is: A first orthoimage creation step creates a provisional orthoimage in which a portion of the road surface is covered by the obstacle, based on a plurality of first images captured in the first shooting step and the three-dimensional coordinates of the first feature points acquired in the first coordinate acquisition step, A shape detection step that detects the shape of at least a portion of the road surface and the portion other than the road surface in the area covered by the obstacle, based on a plurality of second images captured in the second shooting step and the three-dimensional coordinates of the second feature points acquired in the second target acquisition step, The orthoimage creation method according to claim 1, further comprising: a second orthoimage creation step of creating a corrected orthoimage by correcting at least a portion of the area covered by the obstacle in the provisional orthoimage created in the first orthoimage creation step to an area not covered by the obstacle.

3. The orthomosaic image creation method according to claim 1 or 2, characterized in that the first imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude higher than the obstacle, and the second imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude lower than the obstacle or a camera positioned at an altitude lower than the obstacle.

4. The orthomosaic image creation method according to any one of claims 1 to 3, characterized in that in the second shooting step, a plurality of signs are installed on the road surface and the obstacle in the area covered by the obstacle, and the plurality of signs are photographed such that they are included as second feature points in at least two of the second photographed images.

5. A first image storage means for storing multiple first images taken by a first imaging device from an altitude higher than the obstacle, of a road in which a portion of the road surface is covered by an obstacle when viewed from above, A second image storage means for storing a plurality of second images taken by a second imaging device from a height lower than the obstacle, of the area covered by the obstacle, A first coordinate storage means for storing the three-dimensional coordinates of a first feature point located outside the area covered by the aforementioned obstacle and included in at least two of the plurality of first captured images, A second coordinate storage means for storing the three-dimensional coordinates of a second feature point located in the area covered by the aforementioned obstacle and included in at least two of the plurality of second captured images, An orthoimage creation system characterized by comprising: an orthoimage creation means that creates a corrected orthoimage in which at least a portion of the area of ​​the road surface covered by the obstacle is corrected to an area not covered by the obstacle, based on a plurality of first captured images stored in the first captured image storage means; a plurality of second captured images stored in the second captured image storage means; the three-dimensional coordinates of the first feature points stored in the first coordinate storage means; and the three-dimensional coordinates of the second feature points stored in the second storage means.

6. The orthomosaic image creation means is A first orthoimage creation means creates a provisional orthoimage in which a portion of the road surface is covered by an obstacle, based on a plurality of first captured images stored in the first image storage means and the three-dimensional coordinates of the first feature points stored in the first coordinate storage means. A shape detection means for detecting the shape of at least a portion of the road surface and the portion other than the road surface in the area covered by the obstacle, based on a plurality of second captured images stored in the second capture storage means and the three-dimensional coordinates of the second feature points stored in the second coordinate storage means, The orthoimage creation system according to claim 5, further comprising: a second orthoimage creation means for creating a corrected orthoimage by correcting at least a portion of the area covered by the obstacle in the provisional orthoimage created by the first orthoimage creation means to an area not covered by the obstacle.

7. The orthomosaic image creation system according to claim 5 or 6, characterized in that the first imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude higher than the obstacle, and the second imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude lower than the obstacle or a camera positioned at an altitude lower than the obstacle.

8. The orthomosaic image creation system according to any one of claims 5 to 7, characterized in that the plurality of second images stored in the second image storage means are captured such that the road surface and the plurality of signs installed on the obstacle in the area covered by the obstacle are included as second feature points in at least two of the second images.

9. A marker used in the orthomosaic image creation method described in claim 4, A sign characterized by being in the form of a sticker with an adhesive layer formed on its back surface.

10. A marker used in the orthomosaic image creation system according to claim 8, A sign characterized by being in the form of a sticker with an adhesive layer formed on its back surface.

11. A first shooting step involves taking a photograph of a road, where a portion of the road surface is covered by an obstacle when viewed from above, using a first shooting device from an altitude higher than the obstacle, to obtain a plurality of first images. A second imaging step involves photographing the area covered by the aforementioned obstacle from a height lower than the obstacle using a second imaging device to obtain a plurality of second images, A first coordinate acquisition step of acquiring three-dimensional coordinates of a first feature point that is located outside the area covered by the aforementioned obstacle and is included in at least two of the plurality of first captured images, A second coordinate acquisition step of acquiring three-dimensional coordinates of a second feature point that is located in the area covered by the aforementioned obstacle and is included in at least two of the plurality of second captured images, A method for creating a three-dimensional model, comprising: a three-dimensional model creation step, which creates a corrected three-dimensional model in which at least a portion of the area of ​​the road surface covered by the obstacle is corrected to an area not covered by the obstacle, based on a plurality of first images captured in the first shooting step, a plurality of second images captured in the second shooting step, the three-dimensional coordinates of the first feature points acquired in the first coordinate acquisition step, and the three-dimensional coordinates of the second feature points acquired in the second coordinate acquisition step.

12. The aforementioned step of creating a three-dimensional model is: A first creation step involves creating a provisional three-dimensional model in which a portion of the road surface is covered by the obstacle, based on a plurality of first captured images taken in the first shooting step and the three-dimensional coordinates of the first feature points obtained in the first coordinate acquisition step. A shape detection step that detects the shape of at least a portion of the road surface and the portion other than the road surface in the area covered by the obstacle, based on a plurality of second images captured in the second shooting step and the three-dimensional coordinates of the second feature points acquired in the second target acquisition step, The method for creating a three-dimensional model according to claim 11, further comprising a second three-dimensional model creation step of creating a corrected three-dimensional model in which at least a portion of the area of ​​the provisional three-dimensional model created in the first creation step that is covered by the obstacle is corrected to an area that is not covered by the obstacle.

13. The method for creating a three-dimensional model according to claim 11 or 12, characterized in that the first imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude higher than the obstacle, and the second imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude lower than the obstacle, or a camera positioned at an altitude lower than the obstacle.

14. The method for creating a three-dimensional model according to any one of claims 11 to 13, characterized in that in the second shooting step, a plurality of signs are installed on the road surface and the obstacle in the area covered by the obstacle, and the plurality of signs are photographed such that they are included as second feature points in at least two of the second photographed images.

15. A first image storage means for storing multiple first images taken by a first imaging device from an altitude higher than the obstacle, of a road in which a portion of the road surface is covered by an obstacle when viewed from above, A second image storage means for storing a plurality of second images taken by a second imaging device from a height lower than the obstacle, of the area covered by the obstacle, A first coordinate storage means for storing the three-dimensional coordinates of a first feature point located outside the area covered by the aforementioned obstacle and included in at least two of the plurality of first captured images, A second coordinate storage means for storing the three-dimensional coordinates of a second feature point located in the area covered by the aforementioned obstacle and included in at least two of the plurality of second captured images, A three-dimensional model creation system comprising: a three-dimensional model creation means for creating a corrected three-dimensional model in which at least a portion of the area of ​​the road surface covered by the obstacle is corrected to an area not covered by the obstacle, based on a plurality of first captured images stored in the first captured image storage means; a plurality of second captured images stored in the second captured image storage means; the three-dimensional coordinates of the first feature points stored in the first coordinate storage means; and the three-dimensional coordinates of the second feature points stored in the second coordinate storage means.

16. The three-dimensional model creation means is A first three-dimensional model creation means creates a provisional three-dimensional model in which a portion of the road surface is covered by the obstacle, based on a plurality of first captured images stored in the first captured image storage means and the three-dimensional coordinates of the first feature points stored in the first coordinate storage means, A shape detection means for detecting the shape of at least a portion of the road surface and the portion other than the road surface in the area covered by the obstacle, based on a plurality of second captured images stored in the second captured image storage means and the three-dimensional coordinates of the second feature points stored in the second coordinate storage means, The three-dimensional model creation system according to claim 15, further comprising a second three-dimensional model creation means for creating a corrected three-dimensional model in which at least a portion of the area of ​​the provisional three-dimensional model created by the first three-dimensional model creation means that is covered by the obstacle is corrected to an area that is not covered by the obstacle.

17. The three-dimensional model creation system according to claim 15 or 16, characterized in that the first imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude higher than the obstacle, and the second imaging device is an unmanned aerial vehicle or model aircraft flying at an altitude lower than the obstacle or a camera positioned at an altitude lower than the obstacle.

18. The three-dimensional model creation system according to any one of claims 15 to 17, characterized in that the plurality of second images stored in the second image storage means are captured such that the road surface and a plurality of signs installed on the obstacle are included as second feature points in at least two of the second images in the area covered by the obstacle.

19. A marker used in the method for creating a three-dimensional model according to claim 14, A sign characterized by being in the form of a sticker with an adhesive layer formed on its back surface.

20. A marker used in the 3D model creation system described in claim 18, A sign characterized by being in the form of a sticker with an adhesive layer formed on its back surface.

Images