Point cloud processing device and point cloud processing system

Abstract

Provided is a point cloud processing device comprising: a first point cloud generation unit that generates a first three-dimensional point cloud of a target region using a plurality of images captured by a camera comprised by a moving body; and a second point cloud generation unit that generates a second three-dimensional point cloud of a partial region within the target region using an image, among the plurality of images, selected on the basis of information relating to a terminal device set up in the target region.

Description

Point cloud processing device and point cloud processing system

[0001] The present technology relates to a point cloud processing device and a point cloud processing system.

[0002] A conventional technique is to generate a three-dimensional point cloud representing the ground surface from multiple image data captured by a camera mounted on a mobile object such as a drone flying in the sky.

[0003] In generating a three-dimensional point cloud, errors may occur due to the position detection accuracy of a position detection sensor, etc. Therefore, a technology has been proposed in which markers 400 are placed in an area to be generated as a three-dimensional point cloud, and the markers 400 are detected from an image obtained by photographing the area in which the markers 400 are placed, thereby generating a three-dimensional point cloud (Patent Document 1).

[0004] WO2023223887 publication

[0005] The technology described in Patent Document 1 can improve the efficiency of the marker 400 detection work for generating a three-dimensional point cloud, but further improvement in the accuracy of generating a three-dimensional point cloud is required.

[0006] The present technology has been developed in consideration of such problems, and aims to provide a point cloud processing device and a point cloud processing system that can generate another three-dimensional point cloud in order to improve the accuracy of a three-dimensional point cloud.

[0007] In order to solve the above-mentioned problems, the first technology is a point cloud processing device including a first point cloud generation unit that generates a first three-dimensional point cloud of a target area using a plurality of images captured by a camera equipped on a moving object, and a second point cloud generation unit that generates a second three-dimensional point cloud of a partial area within the target area using an image selected from the plurality of images based on information related to a terminal device installed within the target area.

[0008] The second technology is a point cloud processing system comprising a mobile body equipped with a camera, a point cloud processing device equipped with a first point cloud generation unit that generates a first three-dimensional point cloud of a target area using multiple images captured by the camera equipped on the mobile body, and a second point cloud generation unit that generates a second three-dimensional point cloud of a partial area within the target area using images selected from the multiple images based on information about terminal devices installed within the target area.

[0009] The third technology is a point cloud processing device that includes a first point cloud generation unit that generates a first three-dimensional point cloud of a target area using multiple images captured by a camera equipped on a moving body, and a third point cloud generation unit that generates a third three-dimensional point cloud of an area around a marker in the target area using an image selected from the multiple images based on information about a marker installed within the target area.

[0010] The fourth technology is a point cloud processing system comprising a mobile body equipped with a camera, a point cloud processing device equipped with a first point cloud generation unit that generates a first three-dimensional point cloud of a target area using multiple images captured by the camera equipped on the mobile body, and a third point cloud generation unit that generates a third three-dimensional point cloud of an area around a marker in the target area using an image selected from the multiple images based on information about a marker installed in the target area.

[0011] 1 is a diagram showing the configuration of a point cloud processing system 1000 in a first embodiment. FIG. 1 is a diagram showing the movement of a moving body 100 in the point cloud processing system 1000. FIG. 2 is a block diagram showing the configuration of a moving body 100. FIG. 3 is a block diagram showing the configuration of a terminal device 200. FIG. 4 is a block diagram showing the configuration of a point cloud processing device 300 in a first embodiment. FIG. 5 is a diagram showing a three-dimensional point cloud generated by the point cloud processing device 300. FIG. 6 is a diagram showing the relationship between a three-dimensional point cloud without errors and the terminal device 200 in the Z direction. FIG. 7 is a diagram showing the relationship between a three-dimensional point cloud with errors and the terminal device 200 in the Z direction. FIG. 8 is a diagram showing the relationship between a three-dimensional point cloud without errors and the terminal device 200 in the X and Y directions. FIG. 9 is a diagram showing the relationship between a three-dimensional point cloud with errors and the terminal device 200 in the X and Y directions. FIG. 10 is an explanatory diagram of a method for calculating a correction amount in the Z direction. FIG. 11 is an explanatory diagram of a method for calculating a correction amount in the X and Y directions. FIG. 12 is a diagram showing an example of a texture to be applied to the top surface of the terminal device 200. FIG. 13 is a diagram showing the configuration of a point cloud processing system 1000 in a second embodiment. FIG. 14 is a block diagram showing the configuration of a point cloud processing device 300 in a second embodiment. 1 is a diagram showing the relationship between a three-dimensional point cloud without errors and a marker 400 in the X and Y directions. FIG. 2 is a diagram showing the relationship between a three-dimensional point cloud with errors and a marker 400 in the X and Y directions. FIG. 3 is an explanatory diagram of a method for verifying the accuracy of the position of a first three-dimensional point cloud in the X and Y directions. FIG. 4 is an explanatory diagram of the densities of the first and third three-dimensional point clouds. FIG. 5 is an explanatory diagram of an example in which a plurality of markers 400 are installed. FIG. 6 is a diagram showing a GUI for inputting error measurement positions. FIG. 7 is an explanatory diagram of accuracy verification of the first three-dimensional point cloud. FIG. 8 is a diagram explaining the effect of displaying a third three-dimensional point cloud on a GUI. FIG. 9 is a block diagram showing a first modified example of the point cloud processing device 300. FIG. 10 is a block diagram showing a second modified example of the point cloud processing device 300. FIG. 11 is a block diagram showing a third modified example of the point cloud processing device 300. FIG. 12 is a block diagram showing a fourth modified example of the point cloud processing device 300.

[0012] Hereinafter, embodiments of the present technology will be described with reference to the drawings. The description will be made in the following order: <First embodiment> [Configuration of point cloud processing system 1000] [Configuration of mobile body 100] [Configuration of terminal device 200] [Configuration and processing of point cloud processing device 300] <Second embodiment> [Configuration of point cloud processing system 1000] [Configuration and processing of point cloud processing device 300] <Modification>

[0013] First Embodiment [Configuration of Point Cloud Processing System 1000] As shown in FIGS. 1 and 2, the point cloud processing system 1000 is configured by a mobile object 100 and a terminal device 200 having the function of a point cloud processing device 300.

[0014] The mobile object 100 is an unmanned aerial vehicle such as a drone that flies in the sky. The mobile object 100 is equipped with a camera 105 and can fly automatically and take pictures based on a pre-set movement route. The movement route can be set using existing mobile object control software or the like.

[0015] When performing automatic flight and automatic photography, route information, photography position, photography direction, photography timing, etc. are set in advance, and the mobile body 100 controls movement and photography according to the settings. As shown in FIG. 2, the mobile body 100 flies above an area (target area) for which a three-dimensional point cloud is to be created, and periodically photographs the ground surface with a camera 105 during flight to obtain multiple images. In addition to automatic flight, the mobile body 100 may also be capable of flying manually by a pilot. The target area is an area in which the ground surface can be photographed from the air.

[0016] The moving object 100 can take pictures while changing the moving route in response to instructions from the user via wireless communication while moving. Furthermore, the timing of taking pictures can be added or changed in response to instructions from the user via wireless communication while moving.

[0017] 1, the terminal device 200 is installed as a fixed base station within the target area, supported by a tripod or a stand. The terminal device 200 is preferably installed outdoors and not obstructed by other objects so that it can be photographed by the camera 105 of the mobile object 100 and can reliably communicate with a Global Navigation Satellite System (GNSS) satellite using a GNSS device. In this embodiment, the terminal device 200 is described as having a circular shape in a plan view, but the shape of the terminal device 200 is not limited to this and may be any shape.

[0018] The terminal device 200 has a function as a point cloud processing device 300. The point cloud processing device 300 creates a first three-dimensional point cloud, which is a three-dimensional point cloud for the target area, based on multiple images obtained by the camera 105 of the mobile body 100 capturing images of the ground surface of the target area. The point cloud processing device 300 also creates a second three-dimensional point cloud, which is a three-dimensional point cloud for a partial area within the target area, based on multiple images captured by the camera 105 of the mobile body 100 capturing images of the ground surface of the target area. The partial area is an area that includes the terminal device 200 installed within the target area as shown in FIG. 1 and is smaller than the target area. In the embodiment, the target area and the partial area are rectangular areas, but they may be other shapes, such as circular, trapezoidal, or free-form. Note that objects such as buildings and automobiles may be present on the ground surface.

[0019] A 3D point cloud is a set of data consisting of multiple points with location and color information. It can represent terrain, objects, etc. as a large number of point sets, and can be used in a variety of fields, including civil engineering, architecture, and manufacturing.

[0020] In this embodiment, the mobile object 100 is an unmanned aerial vehicle that flies in the sky, and is therefore connected wirelessly to the terminal device 200. However, data and information may be supplied from the mobile object 100 to the terminal device 200 via a recording medium such as a USB flash memory or an SD memory card.

[0021] Wireless connection methods include, for example, Wi-Fi, wireless LAN (Local Area Network), 4G (fourth generation mobile communication system), 5G (fifth generation mobile communication system) networks, Bluetooth (registered trademark), NFC (Near Field Communication), Ethernet (registered trademark), etc.

[0022] [Configuration of moving body 100] The configuration of moving body 100 will be described with reference to Fig. 3. Although not shown, moving body 100 has an exterior configuration including a housing, rotors, and rotor support shafts. However, the rotors are not limited to rotors and may be fixed wings.

[0023] The UAV (Unmanned Aerial Vehicle) control unit 101 is composed of a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), etc., and controls the entire moving body 100 and each part thereof by executing processes and issuing commands according to programs. The UAV control unit 101 also controls the moving speed, moving direction, turning direction, etc. of the moving body 100 by supplying control signals to the actuators 102 that control the output of the actuators 102.

[0024] In addition, the UAV control unit 101 controls the output of the actuator 102 while comparing the current position of the moving body 100 with a predetermined movement route, thereby controlling the moving body 100 to move along the predetermined movement route.

[0025] The actuator 102 is a drive source for driving the rotor, and is provided at the tip of the rotor support shaft, etc. The actuator 102 operates under the control of the UAV control unit 101.

[0026] The gimbal 103 rotatably supports the camera 105 at the bottom of the moving body 100. The gimbal 103 is a type of support that supports the camera 105 on, for example, two or three axes.

[0027] The gimbal control unit 104 controls the operation of the gimbal 103 that supports the camera 105. By controlling the rotation of the axis of the gimbal 103 with the gimbal control unit 104, the attitude of the camera 105 can be freely adjusted.

[0028] The camera 105 includes an imaging element, a signal processing circuit, etc., and captures RGB (Red, Green, Blue) or monochrome images. The imaging element may be a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), etc.

[0029] The camera 105 is mounted, for example, on the bottom surface of the housing of the moving body 100 so as to be suspended via a gimbal 103 in order to photograph the ground surface of the target area. The camera 105 is capable of photographing with its lens pointed in any direction from 360 degrees horizontally to vertically downward by driving the gimbal 103. Note that the camera 105 may be mounted on the moving body 100 in any manner as long as it can photograph with its lens pointed in all directions from 360 degrees horizontally to vertically downward.

[0030] The position information acquisition unit 106 acquires the position (for example, latitude, longitude, and altitude) of the camera 105 using, for example, a GNSS device or a GPS (Global Positioning System) device.

[0031] The attitude information acquisition unit 107 is configured, for example, by an IMU (Inertial Measurement Unit), and acquires the attitudes of the mobile body 100 and the camera 105 using a two-axis or three-axis acceleration sensor, an angular velocity sensor, a gyro sensor, etc. The attitude information is, for example, a three-axis rotation vector (yaw, pitch, roll) calculated from the acceleration and angular velocity of the mobile body 100. Note that the position information and attitude information of the camera 105 can also be acquired using, for example, a laser sensor capable of acquiring three-dimensional information and SLAM (Simultaneous Localization and Mapping) technology based on the output of the laser sensor.

[0032] The sensor control unit 108 controls the acquisition of images by the camera 105 and the acquisition of camera position information by the position information acquisition unit 106 to be performed in synchronization. The sensor control unit 108 also associates the images with the camera position information acquired in synchronization with the image capture. Thus, camera position information is associated with each of the multiple images captured by the camera 105. The sensor control unit 108 may also add time information, such as a timestamp, to the images and camera position information, indicating that the images and camera position information are synchronized.

[0033] The communication unit 109 is a communication terminal or communication module for various purposes, which performs communication processing via a transmission path such as the Internet, and communication with various devices via wired / wireless communication, bus communication, etc. The mobile object 100 transmits images and camera position information to the point cloud processing device 300 via the communication unit 109. Note that the mobile object 100 may transfer this information to the point cloud processing device 300 via an external storage medium such as a USB flash memory or an SD memory card.

[0034] In addition, a recording medium such as a semiconductor memory may be used, and the recording medium may be removed from the mobile body 100 after the mobile body 100 lands, and connected to a recording medium connection slot (not shown) of the point cloud processing device 300, thereby allowing images and camera position information to be transferred.

[0035] For example, a mobile body 100 such as a drone can fly as desired by controlling the output of actuators 102. When hovering in the air, the attitude information acquisition unit 107 or a separately provided gyro sensor detects tilt, and the output of the actuator 102 on the side where the aircraft is lowered is increased and the output of the actuator 102 on the side where the aircraft is raised is decreased, thereby keeping the aircraft horizontal. Furthermore, when moving forward, the output of the actuator 102 in the direction of travel is decreased and the output of the actuator 102 in the opposite direction is increased, thereby causing the aircraft to assume a forward-leaning attitude and generating propulsion in the direction of travel.

[0036] [Configuration of Terminal Device 200] Next, the configuration of the terminal device 200 will be described with reference to FIG.

[0037] The CPU 201 functions as an arithmetic processing unit that performs various processes and controls the entire terminal device 200 and each unit. The CPU 201 executes various processes in accordance with programs stored in the ROM 202 or the nonvolatile memory unit 204, or programs loaded from the storage unit 208 to the RAM 203. The nonvolatile memory unit 204 may be, for example, an EEPROM (Electrically Erasable Programmable Read Only Memory). The RAM 203 also stores data and the like required for the CPU 201 to execute various processes, as appropriate.

[0038] The CPU 201, ROM 202, RAM 203, and nonvolatile memory unit 204 are interconnected via a bus, to which an input / output interface 205 is also connected.

[0039] The input / output interface 205 is connected to a GNSS device 206 , an input unit 207 , a storage unit 208 , a communication unit 209 , and a drive 210 .

[0040] The GNSS device 206 receives information such as the satellite position and signal transmission time transmitted from GNSS satellites in a satellite positioning system, and thereby acquires the three-dimensional coordinates of the terminal device 200 based on the GNSS satellite. By being provided with the GNSS device 206, the terminal device 200 can acquire its own three-dimensional coordinates. The three-dimensional coordinates of the terminal device 200 acquired by the GNSS device 206 are supplied to an image selection unit 305 and a point cloud correction unit 308 of the point cloud processing device 300.

[0041] The input unit 207 is, for example, various types of operators or operation devices such as a keyboard, a mouse, keys, a dial, a touch panel, a touch pad, a remote controller, etc. The input unit 207 detects user operations, and the CPU 201 interprets signals corresponding to the input operations.

[0042] The storage unit 208 is a large-capacity storage medium such as a hard disk, flash memory, etc. The storage unit 208 stores various applications, data, information, etc.

[0043] The communication unit 209 is a communication terminal or communication module for various purposes, which performs communication processing via a transmission path such as the Internet, and communication with various devices via wired / wireless communication, bus communication, etc. The point cloud processing device 300 receives images and camera position information transmitted from the mobile object 100 via the communication unit 209. The point cloud processing device 300 can also transmit a three-dimensional point cloud to the outside via the communication unit 209. The point cloud processing device 300 may transfer information and data between the mobile object 100 and an external device via an external storage medium such as a USB flash memory or an SD memory card.

[0044] A drive 210 is connected to the input / output interface 205 as needed. A removable storage medium 211 is appropriately attached via the drive 210. The removable storage medium 211 includes a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like.

[0045] The drive 210 can read data files such as programs used in various processes from a removable storage medium 211. The read data files are stored in the storage unit 208. Furthermore, the computer programs read from the removable storage medium 211 are installed in the storage unit 208 as needed.

[0046] In the terminal device 200, for example, software for processing the present technology can be installed via network communication by the communication unit 209 or a removable storage medium 211. The software may also be stored in advance in the ROM 202, the storage unit 208, etc. Images captured by the camera 105 and processing results of AI image processing may also be received, and the images and processing results may be stored in the storage unit 208 or the removable storage medium 211.

[0047] The terminal device 200 must have a function capable of acquiring its own three-dimensional coordinates using the GNSS device 206. It must also be installed in a position where it is not obstructed by objects and can be photographed by the camera 105 of the mobile object 100. Furthermore, the terminal device 200 has the role of identifying a partial area within the target area. Therefore, the terminal device 200 must be installed within the target area, and it is preferable that it be installed outdoors.

[0048] The terminal device 200 may be, for example, a surveying device equipped with a GNSS function and a point cloud generation function, which is installed at a civil engineering site or a construction site.

[0049] [Configuration and Processing of Point Cloud Processing Device 300] Next, the configuration of the point cloud processing device 300 that operates in the terminal device 200 and the processing in the point cloud processing device 300 will be described with reference to FIG.

[0050] As a prerequisite for processing in the point cloud processing device 300, first, the moving body 100 moves (flies) above the target area based on a movement route created in advance, and photographs are taken by the camera 105.

[0051] As shown in Fig. 2, the camera 105 repeatedly and continuously captures images at predetermined intervals while the moving object 100 is moving along the pre-travel path. In addition, in synchronization with the image capture by the camera 105, the position information acquisition unit 106 acquires camera position information, which is the position of the camera at the time of image capture. As described above, each of the multiple images captured by the camera 105 is associated with camera position information. The moving object 100 then transmits the multiple images and the camera position information to the terminal device 200.

[0052] The information acquisition unit 301 acquires a plurality of images and camera position information transmitted from the mobile object 100 and received by the terminal device 200 , and supplies them to the feature extraction unit 302 .

[0053] The feature extraction unit 302 extracts feature points from each of the multiple images, such as corners of buildings or objects, boundaries of brightness changes, and boundaries of color changes.

[0054] The feature matching unit 303 identifies the correspondence between the multiple images by matching the feature points in each image extracted by the feature extraction unit 302. The feature matching unit 303 may also identify the correspondence between the multiple images using camera position information, or may identify the correspondence between the multiple images using both the feature points and the camera position information.

[0055] The sparse point cloud generation unit 304 generates a three-dimensional point cloud (three-dimensional sparse point cloud) in which points are sparsely distributed using multiple images whose correspondences have been identified. The sparse point cloud generation unit 304 may also estimate the shooting position of the camera 105 based on the multiple images using SfM (Structure From Motion) or the like.

[0056] The image selection unit 305 selects images for generating a three-dimensional point cloud from the multiple images captured by the camera 105 and supplies them to the first point cloud generation unit 306 and the second point cloud generation unit 307 .

[0057] Based on the camera position information, the image selection unit 305 selects images (target area images) from the multiple images for generating a first three-dimensional point cloud of the target area and supplies the selected images to the first point cloud generation unit 306. Based on the camera position information, the image selection unit 305 supplies the first point cloud generation unit 306 with multiple images from which redundancy has been eliminated, for example, by excluding images captured outside the target area and overlapping images. If there are no such images outside the target area or overlapping images, all of the multiple images captured by the camera 105 are supplied to the first point cloud generation unit 306 as target area images.

[0058] In order for the image selection unit 305 to select images to supply to the first point cloud generation unit 306, camera orientation information may be used in addition to camera position information. For this purpose, the mobile body 100 transmits camera orientation information to the terminal device 200 along with the camera position information. The point cloud processing device 300 may also include an estimation processing unit that estimates the orientation of the camera 105 at the time of image capture based on the image. SfM is a method for determining the orientation of the camera 105 from an image.

[0059] The image selection unit 305 also selects an image (partial area image) for generating a second three-dimensional point cloud of a partial area from the multiple images and supplies the selected image to the second point cloud generation unit 307. The second three-dimensional point cloud is a three-dimensional point cloud of a partial area. The partial area image for generating the second three-dimensional point cloud is an image in which the terminal device 200 is present as a subject among the multiple images captured by the moving body 100.

[0060] The image selection unit 305 selects a partial area image from multiple images using the three-dimensional coordinates of the terminal device 200 supplied from the GNSS device 206. The position where the image was captured can be identified using camera position information acquired in synchronization with the image capture. The position of the terminal device 200 can be identified using the three-dimensional coordinates acquired by the GNSS device 206. Therefore, an image in which the terminal device 200 exists as a subject can be identified based on the three-dimensional coordinates of the terminal device 200 and the camera position information. The image selection unit 305 selects the image in which the terminal device 200 exists as a subject as a partial area image. There may be one partial area image or multiple partial area images.

[0061] Furthermore, the image selection unit 305 can detect the terminal device 200 from multiple images using a terminal device detection AI or a known subject detection method, without using the three-dimensional coordinates of the terminal device 200, and select an image in which the terminal device 200 exists as a subject as a partial region image based on the detection result. In this case, the GNSS device 206 does not need to supply the three-dimensional coordinates of the terminal device 200 to the image selection unit 305 of the point cloud processing device 300.

[0062] The three-dimensional coordinates of the terminal device 200 and the detection results of the terminal device 200 in the multiple images correspond to information about the terminal device 200 in the claims.

[0063] The first point cloud generation unit 306 generates a depth image of the target region based on the target region image supplied from the image selection unit 305, and further generates a first three-dimensional point cloud, which is a dense three-dimensional point cloud of the target region, using the depth image. The first point cloud generation unit 306 supplies the first three-dimensional point cloud to the point cloud correction unit 308.

[0064] The second point cloud generation unit 307 generates a depth image of the partial region based on the partial region image supplied from the image selection unit 305, and uses the depth image to generate a second three-dimensional point cloud that is a dense three-dimensional point cloud of the partial region. By the image selection unit 305 supplying the partial region image to the second point cloud generation unit 307, the second point cloud generation unit 307 can generate a second three-dimensional point cloud that is a three-dimensional point cloud of the partial region in which the terminal device 200 is located. The second point cloud generation unit 307 supplies the second three-dimensional point cloud to the point cloud correction unit 308.

[0065] In this way, in this technology, in addition to a first 3D point cloud for the target region, a second 3D point cloud for a partial region where the terminal device 200 is located is generated, as shown in FIG. 6 . FIG. 6 is a side view of the target region and the partial region shown in FIG. 1 . The first point cloud generation unit 306 and the second point cloud generation unit 307 can generate 3D point clouds using SfM. The second 3D point cloud has higher quality than the first 3D point cloud. High quality means low variation among the multiple points constituting the 3D point cloud (low variance of the point cloud), high accuracy (high average positional accuracy of the point cloud), and high density of points. The second 3D point cloud does not need to satisfy all of these requirements; it may satisfy one, all, or a combination of several of them. The first 3D point cloud and the second 3D point cloud are generated in the same coordinate system.

[0066] When the first point cloud generation unit 306 and the second point cloud generation unit 307 use the same depth image generation algorithm, they can adjust the quality of the point cloud by adjusting the resolution of the depth image and the parameter settings during depth calculation (such as the number of iterations of the iterative calculation process when matching stereo images).

[0067] Specifically, increasing the resolution of the depth image increases the density of the point cloud (increasing processing speed).

[0068] Additionally, the quality of point clouds can be improved by changing the parameter settings during depth calculation. One method for improving the quality of point clouds during depth calculation is to gradually change the matching block size from large to small when calculating the matching cost during stereo matching, a method known as "coarse-to-fine search." Another method is to change the settings for the number of images to be matched in stereo matching (to match more images). Another method is to increase the number of iterations of the matching cost minimization operation, which is a nonlinear optimization operation, during the process of obtaining a disparity map for the entire image by optimizing the stereo matching cost.

[0069] The second 3D point cloud may be denser and of higher quality than the first 3D point cloud by using a depth image generation algorithm in the second point cloud generation unit 307 that is different from the depth image generation algorithm in the first point cloud generation unit 306. For example, the first point cloud generation unit 306 may use an algorithm that requires light processing and has medium accuracy, while the second point cloud generation unit 307 may use an algorithm that requires very heavy processing but has high accuracy and robustness.

[0070] Since the terminal device 200 is installed within the target area, when a three-dimensional point cloud of the target area is generated, the three-dimensional point cloud includes a point cloud corresponding to the top surface of the terminal device 200. Here, the relationship between the three-dimensional point cloud of the target area and the position of the top surface of the terminal device 200 will be described.

[0071] First, the side view direction (Z direction) will be described. As shown in Fig. 7A, the terminal device 200 is supported by a tripod or the like and installed on the ground surface of the target area. When a three-dimensional point cloud is generated for the target area in which the terminal device 200 exists, the three-dimensional point cloud includes a point cloud corresponding to the top surface of the terminal device 200, as shown in Fig. 7B.

[0072] If there is no error in the three-dimensional point cloud (including cases where the error is slight and equal to or less than a predetermined amount), the relationship between the center position of the top surface of the terminal device 200, which can be identified from the three-dimensional coordinates of the terminal device 200 acquired by the GNSS device 206, and the three-dimensional point cloud corresponding to the top surface of the terminal device 200, will be as shown in Figure 7C. Since there is no error, the center position of the top surface of the terminal device 200 and the height in the Z direction of the first three-dimensional point cloud of the terminal device 200 match. It is desirable that the three-dimensional point cloud be error-free and in this state.

[0073] On the other hand, if there is an error in the Z direction in the 3D point cloud, as shown in Figure 8B or 8C, the center position of the top surface of the terminal device 200 will be misaligned with the position in the Z direction of the 3D point cloud of the terminal device 200. Also, a misalignment will occur between the 3D point cloud of the ground surface and the ground surface itself. In this technology, by using the second 3D point cloud, it is possible to correct such a misalignment of the 3D point cloud.

[0074] Next, the planar view directions (X and Y directions) will be described. As shown in Fig. 9A, the terminal device 200 is installed on the ground surface of the target area. When a three-dimensional point cloud is generated for the target area in which the terminal device 200 exists, the three-dimensional point cloud includes a point cloud corresponding to the top surface of the terminal device 200, as shown in Fig. 9B.

[0075] If there is no error in the three-dimensional point cloud (including cases where the error is slight and equal to or less than a predetermined amount), the relationship between the center position of the top surface of the terminal device 200, which can be identified from the three-dimensional coordinates of the terminal device 200 acquired by the GNSS device 206, and the three-dimensional point cloud corresponding to the top surface of the terminal device 200, will be as shown in Figure 9B. Since there is no error, the center position of the top surface of the terminal device 200 coincides with approximately the center of the three-dimensional point cloud of the top surface of the terminal device 200. It is desirable that the three-dimensional point cloud be error-free and in this state.

[0076] On the other hand, if the three-dimensional point cloud has errors in the X and Y directions, as shown in Fig. 10B, the center position of the top surface of the terminal device 200 will be misaligned with the position in the X and Y directions of the three-dimensional point cloud of the terminal device 200. In the present technology, by using the second three-dimensional point cloud, it is possible to correct such misalignment of the three-dimensional point cloud.

[0077] Returning to the description of Fig. 5, the point cloud correction unit 308 then calculates a correction amount for correcting the first three-dimensional point cloud based on the second three-dimensional point cloud and the three-dimensional coordinates of the terminal device 200, and corrects the first three-dimensional point cloud based on the correction amount.

[0078] A method for calculating the correction amount in the Z direction of the first three-dimensional point cloud will be described with reference to Fig. 11 . First, the second three-dimensional point cloud of a partial region shown in Fig. 11A is clipped within a predetermined range. The predetermined range is, for example, a cylindrical range of radius r and height h centered on the center position of the top surface of the terminal device 200, which can be identified from the three-dimensional coordinates of the terminal device 200. As a result, the second three-dimensional point cloud outside the predetermined range is excluded from the processing target, as shown in Fig. 11B .

[0079] 11C , a predetermined value in the Z direction of the clipped second 3D point cloud is determined, such as the average or median of the Z coordinates of the points constituting the clipped second 3D point cloud, or the height of a plane obtained by superimposing a plane parallel to the XY plane on the clipped second 3D point cloud.

[0080] Since the first three-dimensional point cloud and the second three-dimensional point cloud are generated in the same coordinate system, as shown in Figure 11D, the difference between the specified value in the Z direction and the center position of the top surface of the terminal device 200 becomes the Z direction correction amount for correcting the first three-dimensional point cloud.

[0081] By adjusting the Z-direction position of the first three-dimensional point group based on the Z-direction correction amount calculated in this manner, the first three-dimensional point group can be corrected to achieve high accuracy.

[0082] Next, a method for calculating the correction amount in the XY direction of the first three-dimensional point cloud will be described with reference to Fig. 12 . First, the second three-dimensional point cloud of a partial area shown in Fig. 12A is clipped within a predetermined range. The predetermined range is, for example, a range of radius r centered on the center position of the top surface of the terminal device 200, which can be identified from the three-dimensional coordinates of the terminal device 200. As a result, the second three-dimensional point cloud outside the predetermined range is excluded from the processing target, as shown in Fig. 12B .

[0083] Next, as shown in Fig. 12C, predetermined values ​​in the X and Y directions of the clipped second 3D point cloud are determined. The predetermined values ​​may be the average or median of the X and Y coordinates of the multiple points constituting the clipped second 3D point cloud, the center of a circle obtained by projecting the second 3D point cloud onto the X and Y plane and then fitting the circle to the projected points, etc.

[0084] Since the first three-dimensional point cloud and the second three-dimensional point cloud are generated in the same coordinate system, as shown in Figure 12D, the difference between the specified value in the XY direction and the center position of the top surface of the terminal device 200 becomes the correction amount in the XY direction for correcting the first three-dimensional point cloud.

[0085] By adjusting the XY direction position of the first three-dimensional point cloud based on the XY direction correction amount calculated in this way, the first three-dimensional point cloud can be corrected to have high accuracy.

[0086] The point cloud correction unit 308 may correct only the Z direction of the first three-dimensional point cloud, or may correct only the X and Y directions. The user may be allowed to select which direction to correct.

[0087] Then, the output unit 309 outputs the corrected first three-dimensional point cloud to an external device, a cloud server, etc. via a network, etc. The output unit 309 may also output the corrected first three-dimensional point cloud to the storage unit 208 of the terminal device 200, or a storage medium such as a USB flash memory or an SD memory card.

[0088] Furthermore, the output unit 309 may output the correction amount calculated by the point cloud correction unit 308 as information to an external device, etc., instead of or in addition to the corrected first three-dimensional point cloud. Furthermore, the output unit 309 may output the uncorrected first three-dimensional point cloud or the second three-dimensional point cloud to an external device, etc., instead of or in addition to the corrected first three-dimensional point cloud. The output unit 309 may output the first three-dimensional point cloud and the correction amount to an external device, and the external device may correct the first three-dimensional point cloud.

[0089] The processing of the present technology is performed as described above. According to the present technology, it is possible to generate a second 3D point cloud for improving accuracy by correcting a first 3D point cloud, which is a 3D point cloud of the entire target area. Then, it is possible to correct the first 3D point cloud using a correction amount calculated using the second 3D point cloud.

[0090] Since the terminal device 200 is installed as a fixed base station within the target area, by generating a second three-dimensional point cloud using the three-dimensional coordinates of the terminal device 200, there is no need to install a separate marker 400 for generating the second three-dimensional point cloud within the target area.

[0091] In addition, if there is only one point cloud generation unit and an algorithm that prioritizes quality is used for generating depth images in that point cloud generation unit, the point cloud will be dense and of high quality, and the degree of restoration of the shape of the ground surface and terminal device 200 will be higher, but there is a problem that the processing time will be longer.

[0092] Furthermore, if there is only one point cloud generation unit and an algorithm that prioritizes processing speed is used for generating depth images in that point cloud generation unit, the processing time will be shorter, but there is a problem in that the point cloud will become sparse and further variation will occur, resulting in a lower degree of restoration of the shape of the ground surface and the terminal device 200.

[0093] Furthermore, if there is one point cloud generation unit, an algorithm that prioritizes processing speed is used for generating depth images in that point cloud generation unit, and noise filtering is further performed on the depth images, the processing time is shortened and the variation in the point cloud can be reduced, but there is a problem in that the degree of restoration of the shape of the ground surface and the terminal device 200 is reduced.

[0094] Furthermore, if there is one point cloud generation unit, an algorithm that prioritizes processing speed is used for generating depth images in that point cloud generation unit, and noise filtering is further performed on the point cloud, the processing time is shortened and the variation in the point cloud can be reduced, but there is a problem in that the degree of restoration of the shape of the ground surface and the terminal device 200 is reduced.

[0095] In contrast, in the present technology, the point cloud processing device 300 includes two point cloud generation units, a first point cloud generation unit 306 and a second point cloud generation unit 307. It is preferable to use an algorithm that prioritizes processing speed for the depth calculation in the first point cloud generation unit 306, and an algorithm that prioritizes point cloud quality for the depth calculation in the second point cloud generation unit 307. Furthermore, it is preferable for the first point cloud generation unit 306 to perform noise filtering on the generated depth image. This makes it possible to generate a first 3D point cloud and a second 3D point cloud with high quality, little point cloud variation, and a high degree of restoration of the shape of the ground surface and the terminal device 200, in a short processing time.

[0096] The point cloud processing device 300 may include a synthesis processing unit that synthesizes the first three-dimensional point cloud and the second three-dimensional point cloud to generate a synthetic three-dimensional point cloud. The synthetic three-dimensional point cloud is obtained by superimposing the second three-dimensional point cloud on the first three-dimensional point cloud. By displaying the synthetic three-dimensional point cloud, the first three-dimensional point cloud and the second three-dimensional point cloud can be presented together to the user. This allows the user to easily understand the positional relationship between the first three-dimensional point cloud and the second three-dimensional point cloud. When displayed on a three-dimensional point cloud display device, the points may be displayed individually, or the first three-dimensional point cloud and the second three-dimensional point cloud may be synthesized to display a synthetic three-dimensional point cloud. The user may be able to select which three-dimensional point cloud to display.

[0097] The top surface of the terminal device 200 shown in the image captured by the camera 105 of the moving object 100 may be provided with anti-aircraft markers, patterns, textures, etc., as shown in FIGS. 13A to 13H. This improves the accuracy of recognizing the terminal device 200 from the image captured by the camera 105 and improves the accuracy of estimating the position of the terminal device 200. Furthermore, when generating the second three-dimensional point cloud, a depth image of the terminal device 200 can be generated more robustly and with higher accuracy. As a result, the quality of the second three-dimensional point cloud can be further improved. If the position of the terminal device 200 can be estimated with high accuracy from the image, the estimation result may be used instead of the three-dimensional coordinates obtained by the GNSS device 206, or the estimation result may be used in combination with the three-dimensional coordinates obtained by the GNSS device 206. Note that the anti-aircraft markers, patterns, and textures shown in FIG. 13 are merely examples, and any pattern or texture having a random pattern may be used.

[0098] Note that even if the top surface of the terminal device 200 is plain white, this itself becomes a pattern in the image, and therefore the position of the terminal device 200 can be estimated from the image captured by the camera 105 by template matching or AI recognition processing. Furthermore, by painting the top surface of the terminal device 200 in a unique color, the terminal device 200 can be detected from the image captured by the camera 105 by color detection processing. As described above, there are methods for estimating the position of the terminal device 200 from an image and methods for detecting the terminal device 200. However, if a method of applying an anti-aircraft marking, pattern, or texture to the top surface of the terminal device 200 as shown in FIG. 13 is adopted, the accuracy of recognizing the terminal device 200 can be further improved, and the accuracy of estimating the position of the terminal device 200 can be improved.

[0099] The terminal device 200 may be equipped with the functions of the point cloud processing device 300 in advance, or the point cloud processing device 300 and the point cloud processing method may be realized by the terminal device 200 executing a program. The program may be pre-installed in the terminal device 200, or may be distributed by download or storage medium, etc., and installed by a user, etc.

[0100] Alternatively, the terminal device 200 may not have the functionality of the point cloud processing device 300, and an electronic device with information processing and communication functions located outside the target area may function as the point cloud processing device 300. In this case, the mobile object 100 transmits multiple images and camera position information to the electronic device, and the terminal device 200 transmits its own three-dimensional coordinates to the electronic device. The electronic device may be a personal computer, smartphone, tablet terminal, or the like. Alternatively, a cloud server may function as the point cloud processing device 300. However, when the terminal device 200 installed within the target area functions as the point cloud processing device 300, there is an advantage in that a three-dimensional point cloud can be generated on-site in the target area, such as at a civil engineering site or construction site.

[0101] Alternatively, an electronic device having computer functions may execute a program to realize the point cloud processing device 300. The program may be pre-installed in the electronic device, or may be distributed by download or storage medium, and installed by a user.

[0102] In addition, the terminal device 200 may have a function for distributing RTK (Real Time Kinematic) correction information, and may supply RTK correction information to construction machinery, etc., using an RTK correction information distribution service consisting of an external radio or cloud server.

[0103] The point cloud processing device 300 may also transmit the created three-dimensional point cloud to an external device, a cloud server, etc. The external device or cloud server may then convert the data format of the three-dimensional point cloud and transmit it to another external device, cloud server, etc.

[0104] A cloud server is not limited to being configured by a single computer device, but may be configured by a system of multiple computer devices. The multiple computer devices are systemized, for example, by a local area network (LAN). Furthermore, multiple computer devices located in remote locations may be systemized by a virtual private network (VPN) using the Internet or the like. The multiple computer devices may include computer devices as a server group (cloud) available through a cloud computing service.

[0105] Second Embodiment [Configuration of Point Cloud Processing System 1000] Next, a second embodiment of the present technology will be described. In the second embodiment, a marker 400 is placed in a target region as shown in Fig. 14. The marker 400 serves as a reference for a third three-dimensional point cloud generated in the second embodiment.

[0106] The point cloud processing device 300 creates a third 3D point cloud, which is a 3D point cloud for the area around the marker 400 in the target area (marker surrounding area), based on multiple images of the ground surface of the target area captured by the camera 105 equipped on the mobile body 100. The marker surrounding area is an area that includes the marker 400 installed in the target area and is smaller than the target area, as shown in FIG. 14. In FIG. 14, the target area and the marker surrounding area are shown as rectangular areas, but they may be other shapes, such as circular, trapezoidal, or free-form. Note that objects such as buildings and automobiles may be present on the ground surface.

[0107] The marker 400 is installed so that it can be photographed by the camera 105 provided on the moving body 100, and the three-dimensional coordinates of the marker 400 are known.

[0108] The marker 400 may be an airborne beacon with a GPS function that can obtain the three-dimensional coordinates of the marker 400. Alternatively, a general airborne beacon without a GPS function may also be used as the marker 400. Regardless of whether the marker has a GPS function or not, the appearance of the airborne beacon is the same as that shown in FIG. 13 in the first embodiment. However, when using an airborne beacon without a GPS function, it is necessary to measure the three-dimensional coordinates of the airborne beacon in advance using an external device or system. Methods for measuring the three-dimensional coordinates include, for example, SLAM (Simultaneous Localization and Mapping), conventional surveying methods, LiDAR (Light Detection and Ranging), and methods that refer to existing map data. However, any method that can obtain the three-dimensional coordinates of the marker 400 may be used.

[0109] Additionally, objects that already exist within the target area, such as buildings, or objects that a user has placed within the target area, can also be used as markers 400. Because such objects do not have a GPS function, it is necessary to measure their three-dimensional coordinates in advance using an external device or system, such as SLAM, conventional surveying methods, LiDAR, or map data.

[0110] If the marker 400 has a GPS function and a communication function, the marker 400 transmits three-dimensional coordinate information via communication to the point cloud processing device 300. If the marker 400 does not have a GPS function and the three-dimensional coordinates of the anti-aircraft sign are measured in advance using an external device or system, the three-dimensional coordinate information of the marker 400 must be transmitted from the external device or system to the point cloud processing device 300.

[0111] The marker 400 is preferably placed on a flat surface with no or little inclination within the target area, but the marker 400 may be placed anywhere within the target area.

[0112] When the marker 400 is installed as a verification point, the accuracy of the position of the first 3D point cloud can be verified using the third 3D point cloud. Furthermore, when the marker 400 is installed as a GCP (Ground Control Point) or an orientation point, the position of the first 3D point cloud can be corrected using the third 3D point cloud. Therefore, the user needs to specify the purpose in advance, whether the marker 400 is installed for verifying the accuracy of the position of the first 3D point cloud or for correcting the position of the first 3D point cloud. The installation method of the marker 400 is the same whether the marker 400 is used for verifying the accuracy of the position or for correcting the position.

[0113] The configuration and movement of the moving body 100, the capturing of images by the camera 105 provided in the moving body 100, and the transmission of multiple images and camera position information from the moving body 100 to the terminal device 200 are the same as those in the first embodiment. As a premise for processing in the point cloud processing device 300, first, the moving body 100 moves (flies) above a target area based on a movement route created in advance, and captures images with the camera 105.

[0114] The configuration of the terminal device 200 having the function of the point cloud processing device 300 is the same as that of the first embodiment.

[0115] [Configuration and Processing of Point Cloud Processing Device 300] The configuration of the point cloud processing device 300 that operates in the terminal device 200 and the processing in the point cloud processing device 300 will be described with reference to FIG.

[0116] The configurations and processing of the information acquisition unit 301, feature extraction unit 302, feature matching unit 303, sparse point group generation unit 304, and output unit 309 are the same as those in the first embodiment.

[0117] The image selection unit 305 selects images for generating a three-dimensional point cloud based on a plurality of images captured by the camera 105 and supplies the selected images to the first point cloud generation unit 306 and the third point cloud generation unit 310 .

[0118] The image selection unit 305 selects an image (target area image) for generating a first three-dimensional point cloud of the target area, and supplies the image to the first point cloud generation unit 306, in the same manner as in the first embodiment.

[0119] The image selection unit 305 also selects an image (marker peripheral area image) for generating a third three-dimensional point cloud of the marker peripheral area and supplies the selected image to the third point cloud generation unit 310. The third three-dimensional point cloud is a three-dimensional point cloud of the marker peripheral area. The marker peripheral area image for generating the third three-dimensional point cloud is an image in which the marker 400 exists as a subject among multiple images captured by the moving body 100.

[0120] The image selection unit 305 selects a marker surrounding area image from multiple images using the three-dimensional coordinates of the marker 400. The location where the image was captured can be identified using camera position information acquired in synchronization with the image capture. The three-dimensional coordinates of the marker 400 can be acquired using a GPS function provided in the marker 400 or by measuring the location in advance using an external device or system. Therefore, an image in which the marker 400 exists as a subject can be identified based on the three-dimensional coordinates of the marker 400 and the camera position information. The image selection unit 305 selects an image in which the marker 400 exists as a subject as the marker surrounding area image. Note that there may be one or more marker surrounding area images.

[0121] In addition, the image selection unit 305 can detect the marker 400 from multiple images using marker detection AI or a known subject detection method without using the three-dimensional coordinates of the marker 400, and based on the detection results, select an image in which the marker 400 exists as a subject as the marker surrounding area image.

[0122] The three-dimensional coordinates of the marker 400 and the detection results of the marker 400 in the multiple images correspond to information about the marker 400 in the claims.

[0123] The first point cloud generation unit 306 generates a first three-dimensional point cloud, which is a three-dimensional point cloud of the target region, by the same processing as in the first embodiment. The first point cloud generation unit 306 supplies the first three-dimensional point cloud to the output unit 309.

[0124] The third point cloud generation unit 310 generates a depth image of the marker surrounding region based on the marker surrounding region image supplied from the image selection unit 305, and uses the depth image to generate a third 3D point cloud, which is a dense 3D point cloud of the marker surrounding region. By the image selection unit 305 supplying the marker surrounding region image to the third point cloud generation unit 310, the third point cloud generation unit 310 can generate a third 3D point cloud of the marker surrounding region where the marker 400 exists. The third point cloud generation unit 310 supplies the third 3D point cloud to the output unit 309.

[0125] In this way, in the second embodiment, in addition to the first 3D point cloud for the target region, a third 3D point cloud for the marker surrounding region where the marker 400 is located is generated. The first point cloud generation unit 306 and the third point cloud generation unit 310 can generate 3D point clouds using SfM. The third 3D point cloud has higher quality than the first 3D point cloud. High quality means low variation among the multiple points constituting the 3D point cloud (low point cloud variance), high accuracy (high average positional accuracy of the point cloud), and high point density. The third 3D point cloud does not need to satisfy all of these requirements; it may satisfy one, all, or a combination of several of them. The first and third 3D point clouds are generated in the same coordinate system.

[0126] In the second embodiment, as in the first embodiment, when the first point cloud generation unit 306 and the third point cloud generation unit 310 use the same depth image generation algorithm, the quality of the point cloud can be adjusted by the resolution of the depth image and the parameter settings during depth calculation.

[0127] 14, the terminal device 200 is installed within the target area, but it is not essential to generate a second three-dimensional point cloud for a partial area that is an area that includes the terminal device 200. Furthermore, the terminal device 200 may be installed outside the target area.

[0128] Here, the relationship between the three-dimensional point cloud of the target area in the planar view direction (XY direction) and the top surface position of the marker 400 will be described.

[0129] As shown in Fig. 16A, marker 400 is placed on the ground surface of the target area. When a 3D point cloud is generated for the target area in which marker 400 exists, as shown in Fig. 16B, the 3D point cloud includes a 3D point cloud corresponding to the top surface of marker 400 in addition to a 3D point cloud of the ground surface.

[0130] If there is no error in the three-dimensional point cloud (including cases where the error is slight and equal to or less than a predetermined amount), the relationship between the center position of the top surface of marker 400, which can be identified from the three-dimensional coordinates of marker 400, and the three-dimensional point cloud corresponding to the top surface of marker 400, will be as shown in Figure 16B. Because there is no error, the center position of the top surface of marker 400 coincides with approximately the center of the three-dimensional point cloud of the top surface of marker 400. It is desirable for the three-dimensional point cloud to be error-free and in this state.

[0131] On the other hand, if the three-dimensional point cloud has errors in the X and Y directions, the center position of the top surface of the marker 400 will be misaligned with the position of the three-dimensional point cloud of the marker 400 in the X and Y directions, as shown in FIG. 17B.

[0132] Next, a method for verifying the positional accuracy in the XY directions of the first 3D point cloud will be described with reference to Fig. 18. The positional accuracy verification is to verify the degree of error between the first 3D point cloud for the target region and the 3D position of the actual target region.

[0133] First, the third three-dimensional point cloud of the marker surrounding area shown in FIG. 18A is clipped within a predetermined range. The predetermined range is, for example, a range with a side length of s centered at the center position of the top surface of the marker 400, which can be identified from the three-dimensional coordinates of the marker 400. The third three-dimensional point cloud includes a point cloud corresponding to the top surface of the marker 400 and a point cloud of the ground surface. However, by clipping, points that constitute the third three-dimensional point cloud outside the predetermined range are excluded from processing, as shown in FIG. 18B. Note that the Z direction may also be clipped within a predetermined width based on the physical height information of the marker 400. Clipping in the Z direction is also effective in removing outlier noise and the like from the point cloud in the Z direction.

[0134] 18C , predetermined values ​​in the X and Y directions of the clipped third three-dimensional point cloud are calculated. The predetermined value may be the average or median of the X and Y coordinates of the multiple points constituting the clipped third three-dimensional point cloud, the center of a circle obtained by projecting the third three-dimensional point cloud onto the X and Y plane and then fitting the circle to the projected points, the center of gravity or average position of a sequence of points surrounding the center position of the top surface of marker 400, etc. Alternatively, the predetermined value may be the position of the point among the multiple points constituting the third three-dimensional point cloud that is closest to the center position of the top surface of marker 400, which can be identified from the three-dimensional coordinates of marker 400.

[0135] Because the first three-dimensional point cloud and the third three-dimensional point cloud are generated in the same coordinate system, the error between the predetermined values ​​in the X and Y directions of the third three-dimensional point cloud and the center position of the top surface of the marker 400, as shown in FIG. 18D , can be regarded as the error between the first three-dimensional point cloud and the marker 400. Therefore, the accuracy of the position of the first three-dimensional point cloud can be verified by checking the error between the predetermined values ​​in the X and Y directions of the third three-dimensional point cloud and the center position of the top surface of the marker 400. The smaller the error between the predetermined values ​​in the X and Y directions of the third three-dimensional point cloud and the center position of the top surface of the marker 400, the higher the accuracy of the position of the first three-dimensional point cloud.

[0136] By using the error between the predetermined values ​​in the XY directions of the third three-dimensional point cloud obtained in this manner and the center position of the top surface of the marker 400 as the correction amount, the XY direction position of the first three-dimensional point cloud can be corrected, thereby making the first three-dimensional point cloud highly accurate.

[0137] When verifying the positional accuracy of the first three-dimensional point cloud, if the density of the third three-dimensional point cloud around the marker 400 is low, the resolution in the X and Y directions will be coarse and the verification accuracy will be low. By generating a high-density third three-dimensional point cloud for the area around the marker, it is possible to verify and correct the positional accuracy of the first three-dimensional point cloud with high precision.

[0138] In the second embodiment, the output unit 309 outputs the first three-dimensional point cloud, the third three-dimensional point cloud, the three-dimensional coordinates of the marker 400, and the like to an external device, a cloud server, or the like via a network, etc. Then, the external device, the cloud server, or the like can verify the accuracy of the position of the first three-dimensional point cloud and correct it using the method described above.

[0139] The output unit 309 may output the first three-dimensional point cloud, the third three-dimensional point cloud, the three-dimensional coordinates of the marker 400, etc. to the memory unit 208 of the terminal device 200, or a storage medium such as a USB flash memory or an SD memory card.

[0140] The processing of the second embodiment is performed as described above. According to the second embodiment, when the markers 400 are used as verification points, it is possible to perform highly accurate verification of the positional accuracy of the first 3D point cloud, which is a 3D point cloud of the entire target area, regardless of the generation density and without significantly increasing the computational resources. Furthermore, when the markers 400 are used as GCPs or orientation points, it is possible to improve the positional accuracy of the first 3D point cloud for the target area by performing correction.

[0141] Note that the accuracy of position verification varies depending on the density of the three-dimensional point cloud. This point will be explained with reference to FIG. 19. FIGS. 19A to 19C are schematic diagrams showing the first three-dimensional point cloud and the third three-dimensional point cloud, with the points outside the dashed frame being the first three-dimensional point cloud of the ground surface and the points inside the dashed frame being the third three-dimensional point cloud of the marker 400. Also, FIGS. 19A to 19C show partially enlarged views of the third three-dimensional point cloud.

[0142] Fig. 19A shows an example in which the densities of the first and third three-dimensional point clouds are the same or nearly the same. Fig. 19B shows an example in which the densities of the first and third three-dimensional point clouds are the same or nearly the same, but higher than Fig. 19A. Fig. 19C shows an example in which the density of the third three-dimensional point cloud is higher than that of the first three-dimensional point cloud.

[0143] 19 , the "ideal position for measuring the error" is a coordinate position corresponding to the actual center of the top surface in the three-dimensional point cloud coordinate system of the marker 400. The "top surface center position (three-dimensional coordinates) of the marker 400" is the top surface center position of the marker 400 identified from the three-dimensional coordinates. The "position where the error is actually measured" is the predetermined value described above. The predetermined value is a position used to actually measure the position error, and as described above, is the average or median of the XY coordinates of the multiple points constituting the third three-dimensional point cloud, the center of gravity or average position of the point sequence, the position of the point closest to the top surface center position of the marker 400, etc.

[0144] As shown in FIG. 19A , when both the first and third three-dimensional point clouds are low-density, fewer computational resources are required for verifying the positional accuracy, but the verification accuracy is also lower. When both the first and third three-dimensional point clouds are high-density, as shown in FIG. 19B , the distance between the actual error measurement position (predetermined value) and the center position of the top surface of the marker 400 is shorter than when both the first and third three-dimensional point clouds are low-density, as shown in FIG. 19A , resulting in higher verification accuracy of the position of the first three-dimensional point cloud. However, generating both the first and third three-dimensional point clouds at high density increases the processing load and computational resources required for generation. This requires more computational resources and longer processing time. Therefore, as shown in FIG. 19C , by generating the third three-dimensional point cloud at a high density and the first three-dimensional point cloud at a lower density than the third three-dimensional point cloud, the processing load of the three-dimensional point cloud generation can be reduced to the same level as when both the first and third three-dimensional point clouds are low-density. Furthermore, the accuracy of verifying the position of the first 3D point cloud can be made as high as when both the first 3D point cloud and the third 3D point cloud are high density. That is, if the third 3D point cloud is generated at a high density, the verification accuracy can be increased with fewer computational resources even if the first 3D point cloud is generated at a low density. Therefore, highly accurate positional accuracy verification is possible regardless of the generation density of the first 3D point cloud. This is also true for the first embodiment. The content described with reference to FIG. 19 also applies to the accuracy of correcting the position of a 3D point cloud.

[0145] Although FIG. 14 shows one marker 400 placed in the target area, multiple markers 400 may be placed in the target area. When multiple markers 400 are placed, the point cloud processing device 300 needs to be equipped with multiple point cloud generation units to generate third 3D point clouds for the marker surrounding areas of each marker 400. By using the third 3D point clouds for each of the multiple marker surrounding areas, the accuracy of verifying and correcting the position of the first 3D point cloud for the target area can be improved. The multiple markers 400 may be placed anywhere within the target area. For example, as shown in FIG. 20, if the target area is rectangular, it is preferable to place a total of five markers 400 at the four corners and approximately the center of the target area. In this case, the point cloud processing device 300 generates third 3D point clouds for each of the five marker surrounding areas.

[0146] Here, a description will be given of a GUI (Graphical User Interface) when markers 400 are set as verification points and the positional accuracy of the first three-dimensional point cloud is verified using the third three-dimensional point cloud.

[0147] As described above, in order to verify the positional accuracy of the first 3D point cloud using the third 3D point cloud, a position (referred to as an error measurement position) that is actually used to measure the positional error between the first 3D point cloud and the third 3D point cloud is required. As described above, the error measurement position is the average or median of the XY coordinates of the multiple points that make up the third 3D point cloud, the center of gravity or average position of the point sequence, the position of the point closest to the center position of the top surface of the marker 400, etc. The user can also actually specify the error measurement position by input. The GUI is used by the user to input the error measurement position.

[0148] The GUI is displayed on the display device 500. The display device 500 is, for example, a display device such as a liquid crystal display or an organic electroluminescence (EL) display. The display device 500 may be included in the terminal device 200, may be connected to the terminal device 200, may be included in an apparatus that performs accuracy verification processing, or may be connected to an apparatus that performs accuracy verification processing. The display device 500 may also be a device dedicated to GUI display. The display device 500 may have a GUI display processing function, or the terminal device 200 that includes the display device 500 or to which the display device 500 is connected, or the apparatus that performs accuracy verification processing, may have a GUI display processing function.

[0149] As shown in Figure 21A, when the first three-dimensional point cloud is displayed in 2D in the XY direction in the first display mode of the GUI, if the user inputs an instruction to transition to a mode for verifying the accuracy of the first three-dimensional point cloud, the GUI transitions to the second display mode.

[0150] As shown in Fig. 21B, in the second display mode, the third three-dimensional point cloud is displayed in 2D in the X and Y directions. This second display mode makes it easier to visually recognize the center positions of the markers in the third three-dimensional point cloud, i.e., the error measurement positions, as shown in Fig. 21C, making it easier for the user to specify the error measurement positions by input.

[0151] In addition, the third three-dimensional point cloud may be displayed when the user inputs an instruction to transition to a mode for verifying the accuracy of the first three-dimensional point cloud, regardless of the state of the device displaying the third three-dimensional point cloud, not just when the first three-dimensional point cloud is displayed in 2D.

[0152] The input for specifying the error measurement position may be a touch input to the display device 500 if the display device 500 is a touch panel, an input using a cursor superimposed on the third three-dimensional point cloud on the display device 500, an eye-gaze input, or any input method that can specify a position.

[0153] The user does not select any of the points constituting the third three-dimensional point cloud as the error measurement position, but specifies coordinates in the third three-dimensional point cloud. Therefore, the point specified by the user does not have to be a point, but may be a point between two points. The coordinates are X and Y coordinates.

[0154] The GUI may enlarge the third three-dimensional point cloud so that the user can accurately specify the center of the third three-dimensional point cloud as the error measurement position. The enlarged display may be performed by specifying a specific magnification ratio numerically, or, if the display device 500 is a touch panel, the magnification ratio may be changed in response to a pinch-in operation by the user. The display range of the third three-dimensional point cloud may also be adjustable in response to a user input.

[0155] As described above, in the second embodiment, the output unit 309 outputs the first three-dimensional point cloud, the third three-dimensional point cloud, the three-dimensional coordinates of the marker 400, and the like to an external device or a cloud server via a network or the like, thereby enabling the accuracy of the first three-dimensional point cloud to be verified externally. Therefore, by outputting the error measurement positions input by the user together with this information to an external device or a cloud server, the accuracy of the first three-dimensional point cloud can be verified by the external device or the like. Note that the point cloud processing device 300 may have a function for verifying the accuracy of the first three-dimensional point cloud.

[0156] When the error measurement position is input as shown in FIG. 22A, the accuracy of the first three-dimensional point cloud is verified using the input. To verify the accuracy of the first three-dimensional point cloud, the 3D coordinates P=(X Ｐ , Y Ｐ , Z Ｐ ) is required. The XY coordinates (X Ｐ , Y Ｐ ) is specified, so for accuracy verification, the Z coordinate (Z Ｐ ) needs to be calculated.

[0157] The Z coordinate can be calculated, for example, by averaging or median of the Z coordinates of N (e.g., N = 4) neighboring points around the error measurement position input by the user among the multiple points that make up the third three-dimensional point cloud, or by linear interpolation of the Z coordinate according to the XY distance from the neighboring points.

[0158] When the first three-dimensional point cloud is displayed as shown in Figure 23A, if a user looks at the display and tries to specify the center of the first three-dimensional point cloud as the position for error measurement, the density of the point cloud is sparse, so the area formed by the points around the center (shown by the dashed line in Figure 23A) becomes large, resulting in a misalignment between the input position and the center of the first three-dimensional point cloud.

[0159] On the other hand, according to the GUI of the present technology, as shown in FIG. 23B , a third 3D point cloud having a higher density than the first 3D point cloud is displayed, so the area formed by the points around the center (shown by the dashed line in FIG. 23B ) is smaller, thereby improving the accuracy of specifying the error measurement position. Therefore, the user can accurately input the center of the first 3D point cloud as the error measurement position. This allows for more accurate accuracy verification of the first 3D point cloud.

[0160] In verifying the accuracy of the first three-dimensional point cloud, the position error between the first three-dimensional point cloud and the center position of the top surface of the marker 400 is calculated using the 3D coordinate P of the error measurement position on the first three-dimensional point cloud as a reference.

[0161] <Modifications> Although the embodiments of the present technology have been specifically described above, the present technology is not limited to the above-described embodiments, and various modifications based on the technical ideas of the present technology are possible.

[0162] 24 shows a first modified example of the point cloud processing device 300. In the first modified example, the point cloud processing device 300 includes a correction amount calculation unit 311 instead of the point cloud correction unit 308. The correction amount calculation unit 311 calculates a correction amount in the same manner as the point cloud correction unit 308 in the embodiment, and supplies the correction amount to the first point cloud generation unit 306. The first point cloud generation unit 306 then generates a first three-dimensional point cloud while adjusting either or both of the position in the Z direction and the position in the X and Y directions based on the correction amount.

[0163] Furthermore, the first point cloud generation unit 306 may reflect the correction amount calculated by the correction amount calculation unit 311 in the camera position information, and generate a first three-dimensional point cloud based on the corrected camera position information.

[0164] Furthermore, the first point cloud generation unit 306 may generate the first three-dimensional point cloud by reflecting the amount of correction on a depth image generated as intermediate data in the process of generating the three-dimensional point cloud.

[0165] 25 shows a second modified example of the point cloud processing device 300. As in the second modified example, the point cloud processing device 300 may be configured without the point cloud correction unit 308. In this case, the output unit 309 outputs the first 3D point cloud and the second 3D point cloud to an external device or system, and the external device or system calculates the correction amount and corrects the first 3D point cloud.

[0166] FIG. 26 illustrates a third modified example of the point cloud processing device 300. As in the third modified example, the point cloud processing device 300 may include a point cloud synthesis unit 312. The point cloud synthesis unit 312 receives a first 3D point cloud from the first point cloud generation unit 306 and a second 3D point cloud from the second point cloud generation unit 307. The point cloud synthesis unit 310 overlays these multiple 3D point clouds to generate a synthesized 3D point cloud. The synthesized 3D point cloud is useful when presenting multiple 3D point clouds to a user via display. The point cloud processing device 300 of the second embodiment, which includes the first point cloud generation unit 306 and the third point cloud generation unit 310, may also include the point cloud synthesis unit 312. Furthermore, the point cloud processing device 300 may include both the point cloud correction unit 308 and the point cloud synthesis unit 312, or only one of them.

[0167] 27 shows a fourth modified example of the point cloud processing device 300. As in the fourth modified example, the point cloud processing device 300 may include a first point cloud generation unit 306 that generates a first three-dimensional point cloud for a target region, a second point cloud generation unit 307 that generates a second three-dimensional point cloud for a partial region, and a third point cloud generation unit 310 that generates a third three-dimensional point cloud for a region surrounding a marker. The processing of the second point cloud generation unit 307 is the same as in the first embodiment. The processing of the third point cloud generation unit 310 is the same as in the second embodiment. In other words, the first embodiment and the second embodiment may be combined.

[0168] According to the fourth variant, the second three-dimensional point cloud can be used to verify and correct the accuracy of the Z-direction position of the first three-dimensional point cloud, and the third three-dimensional point cloud can be used to verify and correct the accuracy of the X- and Y-direction position of the first three-dimensional point cloud.

[0169] In the fourth modified example, it is necessary to place the terminal device 200 and the marker 400 within the target region. The image selection unit 305 selects a partial region image from the multiple images based on the three-dimensional coordinates of the terminal device 200 and outputs the selected image to the second point cloud generation unit 307. The image selection unit 305 also selects a marker surrounding region image from the multiple images based on the three-dimensional coordinates of the marker 400 and outputs the selected image to the third point cloud generation unit 310.

[0170] The point cloud processing device 300 of the fourth modified example may include a point cloud synthesis unit 312 .

[0171] The point cloud processing device 300 of the second embodiment may include a point cloud verification unit that verifies the accuracy of the positions of the first three-dimensional point cloud using the method described in the embodiment, and a point cloud correction unit that corrects the first three-dimensional point cloud. In this case, the output unit 309 can output the verification results and the corrected first three-dimensional point cloud to an external device or the like.

[0172] The point cloud processing device 300 of the second embodiment may also include a correction amount calculation unit 311 that calculates a correction amount for correcting the first three-dimensional point cloud using the method described in the embodiment, and may output the correction amount as information to an external device, etc. In this case, the first three-dimensional point cloud can be corrected by the external device, etc. The third and fourth modifications may also include one or more of the point cloud verification unit, point cloud correction unit, and correction amount calculation unit 311 described above.

[0173] In the embodiment, the target area is an outdoor area where the ground surface can be photographed from the sky, but it may be any indoor location as long as the mobile body 100 can move and photograph the area. In this case, a positioning method such as a method using an MBS (Metropolitan Beacon System) or a BLE (Bluetooth Low Energy) beacon may be used instead of GNSS to acquire the three-dimensional coordinates of the terminal device 200 indoors.

[0174] The mobile object 100 may be a manned aircraft, a glider, a helicopter, a balloon, an airship, a rocket, or a device that can move along a rail installed in a high position such as the ceiling of a building, other than an unmanned aircraft. Furthermore, the mobile object 100 is not limited to those that move in the sky, but may be an automobile, a motorcycle, a bicycle, personal mobility, an airplane, a ship, a robot, construction machinery, agricultural machinery, or the like.

[0175] The present technology can also be configured as follows. (1) A point cloud processing device including: a first point cloud generation unit that generates a first three-dimensional point cloud of a target area using a plurality of images captured by a camera equipped on a moving object; and a second point cloud generation unit that generates a second three-dimensional point cloud of a partial area within the target area using an image selected from the plurality of images based on information related to a terminal device installed within the target area. (2) The point cloud processing device according to (1), wherein the partial area includes the terminal device and is a smaller area than the target area. (3) The point cloud processing device according to (1) or (2), wherein the second point cloud generation unit generates the second three-dimensional point cloud using an image in which the terminal device is present as a subject, selected from the plurality of images based on three-dimensional coordinates of the terminal device, which is information related to the terminal device. (4) The point cloud processing device according to any of (1) to (3), wherein the second point cloud generation unit generates the second three-dimensional point cloud using an image in which the terminal device is present as a subject, selected from the plurality of images based on a detection result of the terminal device in the image, which is information related to the terminal device. (5) The point cloud processing device according to any one of (1) to (4), further comprising a point cloud correction unit that calculates a correction amount for correcting the first three-dimensional point cloud based on the second three-dimensional point cloud. (6) The point cloud processing device according to (5), wherein the point cloud correction unit calculates the correction amount as a difference between a predetermined value in a Z direction of the second three-dimensional point cloud and a value in the Z direction of the three-dimensional coordinates. (7) The point cloud processing device according to (5) or (6), wherein the point cloud correction unit calculates the correction amount as a difference between a predetermined value in an XY direction of the second three-dimensional point cloud and a value in the XY direction of the three-dimensional coordinates. (8) The point cloud processing device according to any one of (5) to (7), wherein the point cloud correction unit corrects the first three-dimensional point cloud using the correction amount. (9) The point cloud processing device according to (3), wherein the three-dimensional coordinates are acquired by the terminal device based on information received using a satellite positioning system. (10) The point cloud processing device according to (9), wherein the terminal device is installed at a position where it can receive the information from the satellite positioning system. (11) The point cloud processing device according to any one of (1) to (10), wherein the second three-dimensional point cloud has higher quality than the first three-dimensional point cloud.(12) The point cloud processing device according to any one of (1) to (11), wherein the terminal device has a specific pattern on its top surface. (13) The point cloud processing device according to any one of (1) to (12), wherein processing is performed in the terminal device installed within the target area. (14) The point cloud processing device according to any one of (1) to (13), comprising a synthesis processing unit that synthesizes the first three-dimensional point cloud and the second three-dimensional point cloud. (15) A point cloud processing system comprising: a mobile body equipped with a camera; and a point cloud processing device comprising: a first point cloud generation unit that generates a first three-dimensional point cloud of the target area using a plurality of images captured by the camera equipped on the mobile body; and a second point cloud generation unit that generates a second three-dimensional point cloud of a partial area within the target area using an image selected from the plurality of images based on information related to a terminal device installed within the target area. (16) A point cloud processing device comprising: a first point cloud generation unit that generates a first three-dimensional point cloud of a target area using a plurality of images captured by a camera equipped on a moving object; and a third point cloud generation unit that generates a third three-dimensional point cloud of a marker-surrounding area in the target area using an image selected from the plurality of images based on information about a marker installed in the target area. (17) The point cloud processing device according to (16), wherein the marker-surrounding area includes the marker and is smaller than the target area. (18) The point cloud processing device according to (16) or (17), wherein the third point cloud generation unit generates the third three-dimensional point cloud using an image in which the marker exists as a subject, selected from the plurality of images based on the three-dimensional coordinates of the marker, which is information about the marker. (19) The point cloud processing device according to any of (16) to (18), wherein the marker is an anti-aircraft sign having a position information detection function. (20) The point cloud processing device according to any of (16) to (19), wherein the marker is an anti-aircraft sign whose coordinates are known. (21) The point cloud processing device according to any one of (16) to (20), wherein the markers are set as verification points for verifying the accuracy of the first three-dimensional point cloud. (22) The point cloud processing device according to any one of (16) to (21), wherein the markers are set as GCPs (Ground Control Points) or orientation points for correcting the first three-dimensional point cloud.(23) The point cloud processing device according to any one of (16) to (22), further comprising a second point cloud generation unit that generates a second three-dimensional point cloud of a partial area within the target area using an image selected from the plurality of images based on information related to a terminal device installed within the target area. (24) The point cloud processing device according to (16), in which the third three-dimensional point cloud is displayed on a display unit to verify the accuracy of the first three-dimensional point cloud. (25) The point cloud processing device according to (24), in which, while the third three-dimensional point cloud is displayed, a user can specify a position to be used to measure a positional error between the first three-dimensional point cloud and the third three-dimensional point cloud. (26) The point cloud processing device according to (25), in which the position is the center of the marker in the third three-dimensional point cloud. (27) The point cloud processing device according to (25) or (26), in which a Z coordinate of the position is calculated based on X and Y coordinates of the position, and a positional error between the first three-dimensional point cloud and the third three-dimensional point cloud is measured based on the X and Y coordinates and the Z coordinates. (28) The point cloud processing device according to any one of (24) to (27), wherein the third three-dimensional point cloud is displayed on a display unit in response to a user input instructing accuracy verification of the first three-dimensional point cloud. (29) A point cloud processing system comprising: a mobile body equipped with a camera; and a point cloud processing device including: a first point cloud generation unit that generates a first three-dimensional point cloud of a target area using a plurality of images captured by the camera equipped on the mobile body; and a third point cloud generation unit that generates a third three-dimensional point cloud of a region surrounding a marker in the target area using an image selected from the plurality of images based on information about a marker installed within the target area.

[0176] DESCRIPTION OF SYMBOLS 100: Mobile object 105: Camera 200: Terminal device 300: Point cloud processing device 306: First point cloud generation unit 307: Second point cloud generation unit 310: Third point cloud generation unit 308: Point cloud correction unit 400: Marker 1000: Point cloud processing system

Claims

1. A point cloud processing device comprising: a first point cloud generation unit that generates a first three-dimensional point cloud of a target area using a plurality of images captured by a camera provided in a moving body; and a second point cloud generation unit that generates a second three-dimensional point cloud of a partial area within the target area using an image selected based on information about a terminal device installed within the target area among the plurality of images.

2. The point cloud processing device according to claim 1, wherein the partial area includes the terminal device and is an area smaller than the target area.

3. The point cloud processing device according to claim 1, wherein the second point cloud generation unit generates the second three-dimensional point cloud using an image in which the terminal device exists as a subject, the image being selected from the plurality of images based on the three-dimensional coordinates of the terminal device, which are information about the terminal device.

4. The point cloud processing device according to claim 1, wherein the second point cloud generation unit generates the second three-dimensional point cloud using an image in which the terminal device exists as a subject, the image being selected from the plurality of images based on a detection result of the terminal device in the image, which is information about the terminal device.

5. The point cloud processing device according to claim 1, further comprising a point cloud correction unit that calculates a correction amount for correcting the first three-dimensional point cloud based on the second three-dimensional point cloud.

6. The point cloud processing device according to claim 5, wherein the point cloud correction unit calculates, as the correction amount, a difference between a predetermined value in the Z direction of the second three-dimensional point cloud and a value in the Z direction of the three-dimensional coordinates.

7. The point cloud processing device according to claim 5, wherein the point cloud correction unit calculates, as the correction amount, a difference between a predetermined value in the XY direction of the second three-dimensional point cloud and a value in the XY direction of the three-dimensional coordinates.

8. The point cloud processing device according to claim 5, wherein the point cloud correction unit corrects the first three-dimensional point cloud using the correction amount.

9. The point cloud processing device according to claim 3, wherein the three-dimensional coordinates are acquired by the terminal device based on information received using a satellite positioning system.

10. The point cloud processing device according to claim 9, wherein the terminal device is installed at a position where it can receive the information in the satellite positioning system.

11. The point cloud processing device according to claim 1, wherein the second three-dimensional point cloud is of higher quality than the first three-dimensional point cloud.

12. The point cloud processing device according to claim 1, wherein the terminal device has a specific pattern on its top surface.

13. The point cloud processing device according to claim 1, which performs processing on the terminal device installed within the target area.

14. The point cloud processing apparatus according to claim 1, further comprising a synthesis processing unit that synthesizes the first three-dimensional point cloud and the second three-dimensional point cloud.

15. A point cloud processing system comprising: a mobile body equipped with a camera; a first point cloud generation unit that generates a first three-dimensional point cloud of a target area using a plurality of images captured by the camera of the mobile body; and a second point cloud generation unit that generates a second three-dimensional point cloud of a partial area within the target area using an image selected based on information about a terminal device installed within the target area among the plurality of images.

16. A point cloud processing apparatus comprising: a first point cloud generation unit that generates a first three-dimensional point cloud of a target area using a plurality of images captured by a camera of a mobile body; and a third point cloud generation unit that generates a third three-dimensional point cloud of an area around a marker within the target area using an image selected based on information about the marker installed within the target area among the plurality of images.

17. The point cloud processing apparatus according to claim 16, wherein the area around the marker includes the marker and is an area smaller than the target area.

18. The point cloud processing apparatus according to claim 16, wherein the third point cloud generation unit generates the third three-dimensional point cloud using an image selected from the plurality of images based on the three-dimensional coordinates of the marker, which is the information about the marker, and in which the marker exists as a subject.

19. The point cloud processing apparatus according to claim 16, wherein the marker is an air target with a position information detection function.

20. The point cloud processing apparatus according to claim 16, wherein the marker is an air target with known coordinates.

21. The point cloud processing apparatus according to claim 16, wherein the marker is installed as a verification point for verifying the accuracy of the first three-dimensional point cloud.

22. The point cloud processing apparatus according to claim 16, wherein the marker is installed as a GCP (Ground Control Point) or a calibration point for correcting the first three-dimensional point cloud.

23. The point cloud processing apparatus according to claim 16, further comprising a second point cloud generation unit that generates a second three-dimensional point cloud of a partial area within the target area using an image selected based on information about a terminal device installed within the target area among the plurality of images.

24. The point cloud processing apparatus according to claim 16, wherein the third three-dimensional point cloud is displayed on a display device for verifying the accuracy of the first three-dimensional point cloud.

25. The point group processing apparatus according to claim 24, wherein in a state where the third three-dimensional point group is being displayed, it is possible to specify a position used to measure a positional error between the first three-dimensional point group and the third three-dimensional point group by a user.

26. The point group processing apparatus according to claim 25, wherein the position is the center of the marker in the third three-dimensional point group.

27. The point group processing apparatus according to claim 25, wherein a Z coordinate of the position is calculated based on XY coordinates of the position, and a positional error between the first three-dimensional point group and the third three-dimensional point group is measured based on the XY coordinates and the Z coordinate.

28. The point group processing apparatus according to claim 24, wherein the third three-dimensional point group is displayed on the display device in response to an input instructing accuracy verification of the first three-dimensional point group from a user.

29. A point group processing system comprising: a moving body equipped with a camera; a first point group generation unit that generates a first three-dimensional point group of a target region using a plurality of images captured by the camera equipped with the moving body; and a third point group generation unit that generates a third three-dimensional point group of a region around a marker in the target region using an image selected based on information about a marker installed in the target region among the plurality of images.

Images