Drawing creation system and drawing creation method

Abstract

To provide a system and a method for preparing drawings with which two-dimension drawings can be easily prepared by accurately plotting graphics inside a point group.SOLUTION: In the drawing creation system 1, when a predetermined switching key is input, the switching terminal 107 of the mobile control device 10 switches to a predetermined image screen in which an object image indicating an object in a captured image is pasted on a point cloud on the basis of a plurality of captured images of different viewpoints with respect to the object and an arbitrary point in the point cloud is displayed in a specifiable manner via the object image. When a predetermined point for a figure is designated as a figure point on the point cloud screen or the image screen, the drawing control unit 108 acquires the three dimensional coordinates of the figure point and draws a figure based on the three dimensional coordinates of the figure point. A preparation control part 109 prepares the two-dimension drawings viewed in the geographical coordinates system based on the three dimensional coordinates of the graphic points when the preparation key of the prescribed two-dimension drawings is inputted after the graphic is plotted.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a drawing creation system and a drawing creation method. [Background technology]

[0002] In addition to surveying techniques using 3D surveying instruments (total stations), photogrammetry and laser surveying techniques have also been used. In recent years, with the development of 3D laser scanners that acquire point clouds, surveying techniques that use these 3D laser scanners and cameras have emerged.

[0003] For example, the applicant has already filed and obtained patents for inventions that utilize three-dimensional laser scanners and cameras, as shown in Patent Publication No. 6973726 (Patent Document 1), Patent Publication No. 7057578 (Patent Document 2), Patent Publication No. 7143001 (Patent Document 3), and Patent Publication No. 7250296 (Patent Document 4).

[0004] On the other hand, there is also a technology that combines a point cloud acquired by a 3D laser scanner with a photographic image acquired by a camera.

[0005] For example, Japanese Patent Laid-Open Publication No. 2016-4486 (Patent Document 5) discloses an information processing device having a distance image acquisition unit, a color image acquisition unit, a characteristic location detection unit, a calibration unit, and a registration processing unit. The distance image acquisition unit acquires at least two distance images of an object to be observed, and the color image acquisition unit acquires color images of the object to be observed corresponding to each distance image. The distance images correspond to point clouds. The characteristic location detection unit detects characteristic locations from the acquired color images, and the calibration unit performs a calibration process to associate each pixel in the color image with each point in the distance image corresponding to each pixel, thereby generating calibration information indicating each point corresponding to each pixel. The registration processing unit then uses the detected characteristic locations and the calibration information to align the distance images so that each characteristic location overlaps with each other. This enables stable position and orientation estimation even when features extracted from 3D model data of the object to be observed significantly differ from features that can be extracted from the object to be observed captured in the captured image.

[0006] Furthermore, JP 2022-516298 A (Patent Document 6) describes a method for 3D reconstructing an object, comprising: A) generating multiple images of the object using at least one camera, particularly from different camera angles and positions; B) extracting features of the object from the multiple images; and C) generating a 3D point cloud arranged on a 3D model representing the object. Each of the 3D points corresponds to a feature extractable from at least two of the multiple images, and D) selecting a region from one of the multiple images that includes a projection of at least a portion of a predetermined 3D geometric structure as a building block of the object. The predetermined 3D geometric structure is expressible by a mathematical equation including multiple unknown coefficients, and a subset of the object's features, each corresponding to one of the 3D points, is included in the region, and the number of features in the subset of features is equal to or greater than the number of unknown coefficients in the mathematical equation. The method further includes the steps of: E) identifying each image that includes at least one feature in the subset of features; F) determining a first set of 3D points corresponding to the subset of features selected in step D and a second set of 3D points including at least one 3D point that corresponds to at least one feature not included in the subset of features but that can be extracted from at least two of the images identified in step E; G) using the first set of 3D points and the second set of 3D points, to determine a mathematical equation corresponding to a predetermined 3D geometric structure; and H) rendering a 3D model of the object using at least the predetermined 3D geometric structure, thereby enabling faster and more accurate 3D reconstruction of the object. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent No. 6973726 [Patent Document 2] Patent No. 7057578 [Patent Document 3] Patent No. 7143001 [Patent Document 4] Patent No. 7250296 [Patent Document 5] Japanese Patent Application Laid-Open No. 2016-4486 [Patent Document 6] Special Publication No. 2022-516298 Summary of the Invention [Problem to be solved by the invention]

[0008] In various fields, such as construction, civil engineering, forestry, surveying, and disaster prevention, there is a growing need to collect and utilize the three-dimensional coordinates of point clouds obtained by three-dimensional laser scanners.

[0009] However, currently, the above-mentioned fields are based on two-dimensional drawings such as floor plans and cross sections. For example, when a user creates a two-dimensional drawing of an object such as a building or structure, the user goes to the site, selects measurement points such as the four corners of the object, measures the three-dimensional coordinates of the measurement points using a three-dimensional surveying instrument, and then returns to the office to create a two-dimensional drawing representing the object's contours, outline, and other shapes using the three-dimensional coordinates of the measurement points. This two-dimensional drawing is a DXF (Drawing Exchange Format) file. This method of creating two-dimensional drawings is time-consuming and labor-intensive, and there is a risk of human error, such as the user selecting the wrong measurement points on-site, which reduces the accuracy of the created two-dimensional drawing.

[0010] On the other hand, 3D laser scanners are based on point clouds with 3D coordinates. Therefore, even if a user acquires a point cloud of an object on-site, the 3D coordinates of the point cloud must be converted into 2D coordinates for a 2D drawing of the object, making it difficult for the user to create a 2D drawing using the point cloud as is.

[0011] Given this background, there has been a demand for a method that allows users to add the shape of an object to the three-dimensional coordinates of a point cloud and create a two-dimensional drawing in order to improve work efficiency.

[0012] Here, the technology described in Patent Document 1 can create cross-sectional views, and the technology described in Patent Document 2 can create plot boundaries, but it is not possible to create the above-mentioned two-dimensional drawings using point clouds.Furthermore, the technology described in Patent Document 3 is a technology for converting a point cloud coordinate system of three-dimensional coordinates of a point cloud into a geographic coordinate system, and the technology described in Patent Document 4 is a technology for associating a point cloud of an object with a photographed image, but these technologies are not possible to create the above-mentioned two-dimensional drawings using point clouds.

[0013] Furthermore, the technology described in Patent Document 5 is a technology related to the alignment of point clouds of distance images, and the technology described in Patent Document 6 is a technology for 3D reconstruction of an object, and these technologies cannot be used to create two-dimensional drawings using point clouds.

[0014] Therefore, the present invention has been made to solve the above-mentioned problems, and aims to provide a drawing creation system and a drawing creation method that can accurately draw shapes within a point cloud and easily create two-dimensional drawings. [Means for solving the problem]

[0015] A drawing creation system according to the present invention is a drawing creation system equipped with a mobile terminal device on which a 3D laser scanner and a camera are installed facing the same direction, and includes a point cloud acquisition control unit, an image acquisition control unit, a correspondence control unit, a conversion control unit, a display specification control unit, a switching control unit, a drawing control unit, and a creation control unit. The point cloud acquisition control unit acquires three-dimensional coordinates of a point cloud of the object in a point cloud coordinate system by scanning the object with the 3D laser scanner. The image acquisition control unit acquires a captured image of the object by photographing the object with the camera. The correspondence control unit maps the point cloud coordinate system of the point cloud to the camera coordinate system of the captured image. The conversion control unit acquires three-dimensional coordinates of the current position of the mobile terminal device in a geographic coordinate system using a position communication device attached to the mobile terminal device, and converts the point cloud coordinate system of the point cloud to the geographic coordinate system. The display specification control unit displays the acquired point cloud on a predetermined point cloud screen and displays any point in the acquired point cloud on the point cloud screen so that it can be specified. When a predetermined switching key is input, the switching control unit pastes an object image showing the object in the captured image onto the point cloud based on multiple captured images of the object from different viewpoints, and switches to a predetermined image screen that displays any point in the point cloud via the object image so that any point in the point cloud can be specified.When a predetermined point for a figure is specified as a figure point on the point cloud screen or the image screen, the drawing control unit acquires the three-dimensional coordinates of the figure point and draws the figure based on the three-dimensional coordinates of the figure point.After the figure has been drawn, when a predetermined two-dimensional drawing creation key is input, the creation control unit creates a two-dimensional drawing in a geographic coordinate system based on the three-dimensional coordinates of the figure point.

[0016] The drawing creation method according to the present invention is a drawing creation method for a drawing creation system equipped with a mobile terminal device on which a 3D laser scanner and a camera are installed facing the same direction, and includes a point cloud acquisition control process, an image acquisition control process, a correspondence control process, a conversion control process, a display designation control process, a switching control process, a drawing control process, and a creation control process. Each control process of the drawing creation method corresponds to a respective control unit of the drawing creation system. [Effects of the Invention]

[0017] According to the present invention, it is possible to accurately draw a figure within a point cloud and easily create a two-dimensional drawing. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a schematic diagram illustrating an example of a drawing creation system according to an embodiment of the present invention. [Figure 2] 1 is a flowchart showing the procedure for executing a drawing creation method according to an embodiment of the present invention. [Figure 3] FIG. 3A shows an example of a scan start screen, and FIG. 3B shows an example of acquiring a point cloud with a three-dimensional laser scanner and capturing an image with a camera. [Figure 4] FIG. 4A is a diagram showing an example of the correspondence between a point cloud and a captured image, and FIG. 4B is a diagram showing an example of obtaining three-dimensional coordinates of the current position of a mobile terminal device using a position communication device. [Figure 5] 5A is a diagram showing an example of a case where a point cloud is converted into mesh data, and FIG. 5B is a diagram showing an example of a point cloud screen. [Figure 6] FIG. 6A shows an example of restoring the position and orientation of a camera and the three-dimensional structure of an object from multiple captured images, and FIG. 6B shows an example of matching the three-dimensional structure of an object with a point cloud of the object. [Figure 7] 7A is a diagram showing an example of attaching an object image to a point cloud, and FIG. 7B is a diagram showing an example of an image screen. [Figure 8] 10A and 10B are diagrams illustrating an example of acquiring three-dimensional coordinates of points in a point cloud by specifying the start point of a line segment; [Figure 9] 10A and 10B are diagrams illustrating an example of a case where the three-dimensional coordinates of points in a point cloud are acquired by specifying the end point of a line segment, and the line segment is drawn. [Figure 10] 10A and 10B are diagrams illustrating an example of a case where a line segment is drawn by acquiring three-dimensional coordinates of a corresponding point from a specified end point of the line segment at a position where no point of the point cloud exists. [Figure 11]10A and 10B are diagrams showing an example of creating a plan view and a front view from the three-dimensional coordinates of figure points; [Figure 12] 10A and 10B are diagrams illustrating an example of creating a plan view and a front view from the three-dimensional coordinates of a figure point at a position where no point in the point cloud exists. [Figure 13] 13A is a diagram showing an example of a type of figure, and FIG. 13B is a diagram showing an example of a rectangle drawn by connecting the four corners of a door. [Figure 14] 14A is a diagram showing an example of acquiring a point cloud and a captured image in an embodiment, and FIG. 14B is a diagram showing an example of a point cloud screen and an image screen. [Figure 15] 15A is a diagram showing an example of specifying a figure point on an image screen, and FIG. 15B is a diagram showing an example of drawing a line segment and a broken line on a point cloud screen. [Figure 16] FIG. 16A shows an example of drawing a rectangle on a point cloud screen, and FIG. 16B shows an example of drawing a line segment and a circle on a point cloud screen, and creating a plan view. [Figure 17] FIG. 10 is a diagram showing an example of a case where a two-dimensional plan view and a neighborhood map are displayed in an overlapping manner. DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings to help understand the present invention. Note that the following embodiment is an example of the present invention, and is not intended to limit the technical scope of the present invention.

[0020] As shown in Fig. 1, a drawing creation system 1 according to the present invention includes a mobile terminal device 10, a three-dimensional laser scanner 11, a camera 12, and a position communication device 13. In Fig. 1, the three-dimensional laser scanner 11 and the camera 12 are each attached to the mobile terminal device 10, but they may also be connected to the mobile terminal device 10 wirelessly or via a wire to work together.

[0021] Here, the mobile terminal device 10 includes a display unit that displays a screen, a reception unit (input unit) that receives input of predetermined command keys by user operation, a storage unit that stores data, a control unit that controls each unit, an output unit that outputs data, and a communication unit. Examples of the mobile terminal device 10 include a tablet-type terminal device, a portable notebook computer, and a mobile terminal device (smartphone) with a touch panel.

[0022] The three-dimensional laser scanner 11 also includes a laser irradiation unit, a scattered light detection unit, and a point cloud calculation unit. The laser irradiation unit emits a pulsed laser toward the object. The scattered light detection unit detects scattered light from the laser irradiated onto the object. Based on the detected scattered light, the point cloud calculation unit calculates three-dimensional data such as the distance from the three-dimensional laser scanner 11 to the object and a point cloud of positions on the surface of the object where the scattered light is scattered.

[0023] Here, when the three-dimensional laser scanner 11 is attached to the mobile terminal device 10, it can be, for example, a Lidar (Light Detection and Ranging, Laser Imaging Detection and Ranging) sensor. The three-dimensional point cloud data acquired by the three-dimensional laser scanner 11 is a collection of three-dimensional coordinates of multiple points included in the point cloud. Furthermore, when the three-dimensional laser scanner 11 is connected to the mobile terminal device 10, it can be a standalone three-dimensional laser scanner independent of the mobile terminal device 10.

[0024] Furthermore, camera 12 is basically a visible light camera, and any type of camera can be used as long as it is capable of taking digital photographs. In FIG. 1, 3D laser scanner 11 and camera 12 are attached to mobile terminal device 10 and facing the same direction. Therefore, when 3D laser scanner 11 and camera 12 are pointed at a predetermined object, 3D laser scanner 11 and camera 12 can obtain 3D point cloud data and 2D images of the same object. The captured images can be still images or videos. Furthermore, when camera 12 is connected to mobile terminal device 10, it can also be a standalone camera 12 independent of mobile terminal device 10.

[0025] Furthermore, the position communication device 13 is not particularly limited, and may be, for example, a GPS function attached to a mobile terminal device or a mobile station equipped with an antenna. The position communication device 13 can calculate the current position of the mobile terminal device 10 (position communication device 13) in a geographic coordinate system. Here, the geographic coordinate system is a three-dimensional coordinate system based on a predetermined geographic reference position, and is used in various fields such as construction, civil engineering, forestry, surveying, and disaster prevention. Furthermore, the position communication device 13 may be of any type as long as it can acquire position information of the mobile terminal device 10, and examples of the position communication device 13 include a GPS receiver (single-frequency GNSS receiver) and a dual-frequency multi-GNSS receiver.

[0026] The mobile terminal device 10 incorporates a CPU, ROM, RAM, HDD, SSD, etc. (not shown), and the CPU uses, for example, the RAM as a work area to execute programs stored in the ROM, HDD, SSD, etc. Furthermore, each control unit, which will be described later, is realized by the CPU executing a program.

[0027] Next, the configuration and execution procedure according to an embodiment of the present invention will be described with reference to Figures 1 to 13. First, a surveyor carries a portable terminal device 10 equipped with a 3D laser scanner 11 and a camera 12, visits a predetermined object O (for example, the interior of a building) that he or she wishes to survey, and starts a predetermined application on the portable terminal device 10. Then, the selection reception control unit 101 of the portable terminal device 10 receives input of a start key to start scanning (Figure 2: S101).

[0028] Here, the method of acceptance by the selection acceptance control unit 101 is not particularly limited. For example, the selection acceptance control unit 101 displays a scan start screen on the display unit of the mobile terminal device 10. As shown in FIG. 3A , the scan start screen 300 displays a message 301 ("Start Scan") prompting the user to start scanning, a message 302 ("Please specify a name") prompting the user to specify a name for the data save destination, a name specification field 303, an auto key 304 for automatic scanning, a texture key 305 for simultaneously capturing texture images, and a cancel key 306. Here, texture capture refers to simultaneously capturing images using the camera 11. When selected, each of the auto key 304 and the texture key 305 also functions as a start key.

[0029] The surveyor then specifies a predetermined name 303a (e.g., "AAA"), and as shown in Figure 3B, the surveyor points the laser irradiation unit of the three-dimensional laser scanner 11 at the object O (e.g., near a door inside a building) and selects the texture key 305.The selection reception control unit 101 then accepts the selection of the name 303a ("AAA") and the texture key 305, and accepts the input of the start key.

[0030] Now, once the selection reception control unit 101 has completed its reception, the point cloud acquisition control unit 102 of the mobile terminal device 10 then scans the object O with the three-dimensional laser scanner 11 to obtain the three-dimensional coordinates of the point cloud P of the object O in the point cloud coordinate system (Figure 2: S102).

[0031] Here, there are no particular limitations on the acquisition method of the point cloud acquisition control unit 102. For example, upon receiving input from a start key, the point cloud acquisition control unit 102 starts irradiating the laser of the three-dimensional laser scanner 11. The irradiated laser is reflected by a part of the object O facing the three-dimensional laser scanner 11 and scattered as scattered light. Then, a scattered light detection unit of the three-dimensional laser scanner 11 detects the scattered light, and a point cloud calculation unit of the three-dimensional laser scanner 11 acquires the three-dimensional coordinates (three-dimensional point cloud data) of points in the point cloud P at positions where the scattered light is scattered. Here, the three-dimensional coordinates of the points in the point cloud P include points at positions where the scattered light has returned to the three-dimensional laser scanner 11, but do not include points at positions where the scattered light has not returned to the three-dimensional laser scanner 11. Furthermore, the three-dimensional coordinates (xp1, yp1, zp1) of a specific point P1 included in the three-dimensional coordinates of the point cloud P are constructed in a three-dimensional coordinate system (point cloud coordinate system) with the three-dimensional coordinates (xp0, yp0, zp0) of the point cloud reference position P0 of the three-dimensional laser scanner 11 (for example, the laser irradiation portion of the three-dimensional laser scanner 11) as the reference (origin).

[0032] Now, once the point cloud acquisition control unit 102 has completed acquisition, the image acquisition control unit 103 of the mobile terminal device 10 then acquires a captured image I of the object O by photographing the object O with the camera 12 (Figure 2: S103).

[0033] Here, there is no particular limitation on the acquisition method of the image acquisition control unit 103, but for example, as shown in Fig. 3B, the surveyor points the camera 12 at the object O together with the three-dimensional laser scanner 11, and the image acquisition control unit 103 captures a part of the object O and acquires a captured image I (two-dimensional image) showing that part of the object O. Note that, although the captured image I will be described here as a still image, it may also be a moving image.

[0034] Here, the three-dimensional laser scanner 11 and the camera 12 are each attached to the mobile terminal device 10, and since the three-dimensional laser scanner 11 and the camera 12 face in the same direction, the point cloud P acquired by the three-dimensional laser scanner 11 includes the object O contained in the captured image I acquired by the camera 12.

[0035] In addition, in the present invention, the three-dimensional laser scanner 11 and the camera 12 may be separate units independent of the mobile terminal device 10, so for example, the three-dimensional laser scanner 11 can be configured to face in one direction of the object O and acquire the point cloud P, and the camera 12 of the mobile terminal device 10 can be configured to face in one direction of the object O, so that the three-dimensional laser scanner 11 and the camera 12 are facing in the same direction.

[0036] Once the image acquisition control unit 103 has completed the acquisition, the correspondence control unit 104 of the mobile terminal device 10 then associates the geographic coordinate system of the point group P with the camera coordinate system of the captured image I (FIG. 2: S104).

[0037] Here, there are no particular limitations on the correspondence method of correspondence control unit 104. For example, regarding the relationship between point cloud P of object O and captured image I, if 3D laser scanner 11 and camera 12 attached to mobile terminal device 10 are facing in the same direction, as shown in Fig. 4A, the three-dimensional coordinates (xp1, yp1, zp1) of an arbitrary point P1 among point cloud P in the point cloud coordinate system can be associated with the two-dimensional coordinates (vq1, uq1) of corresponding point Q1 among the point cloud in captured image I in the camera coordinate system based on transformation information such as the shooting positions and attitude information of 3D laser scanner 11 and camera 12, the focal length of camera 12, and a calibration matrix.

[0038] Here, the point cloud coordinate system has a predetermined point cloud reference position P0 as its origin, and, for example, the horizontal axis is the x-axis, the vertical axis is the y-axis, and the depth axis (field of view axis) is the z-axis. The z-axis refers to the axis in the direction from the three-dimensional laser scanner 11 toward the object O. Furthermore, the camera coordinate system has a predetermined point O in the captured image I as its origin, and the horizontal axis is the v-axis and the vertical axis is the u-axis. The captured image I is located perpendicular to the z-axis, at a position a focal length f away from the center of the camera 12 in the z-axis direction toward the object O. Here, the x-axis direction is, for example, the horizontal direction (left-right direction) of the point cloud coordinate system, the z-axis direction is the depth direction (front-back direction) of the point cloud coordinate system, and the y-axis direction is the up-down direction (vertical direction) of the point cloud coordinate system.

[0039] In this way, an arbitrary point P1 in the point cloud P of the object O and a corresponding point Q1 in the captured image I can be associated with each other in the point cloud coordinate system and the camera coordinate system. Therefore, for example, the correspondence control unit 104 uses the above-mentioned conversion information to convert the three-dimensional coordinates (xp, yp, zp) of an arbitrary point P in the point cloud P in the point cloud coordinate system into the two-dimensional coordinates (vq, uq) of the corresponding point Q in the point cloud in the captured image I in the camera coordinate system. This makes it possible to display the converted two-dimensional coordinates (vq, uq) of the corresponding point Q in the captured image I.

[0040] Now, once the correspondence control unit 104 has completed the correspondence, the conversion control unit 105 of the mobile terminal device 10 then uses the position communication device 13 attached to the mobile terminal device 10 to obtain the three-dimensional coordinates (xr0, yr0, zr0) of the current position R0 of the mobile terminal device 10 in the geographic coordinate system, and converts the point cloud coordinate system of the point cloud P into the geographic coordinate system (Figure 2: S105).

[0041] Here, there is no particular limitation on the conversion method of the conversion control unit 105. For example, as shown in Fig. 4B, the conversion control unit 105 acquires three-dimensional coordinates (xr0, yr0, zr0) of the current position R0 of the mobile terminal device 10 in a geographic coordinate system on-site via a position communication device 13 attached to (built-in) the mobile terminal device 10. The geographic coordinate system has a predetermined geographic reference position as its origin, and corresponds to, for example, a point cloud coordinate system, with the horizontal axis being the x-axis, the vertical axis being the y-axis, and the depth axis being the z-axis.

[0042] Next, the conversion control unit 105 subtracts the three-dimensional coordinates (xr0, yr0, zr0) of the current position R0 of the mobile terminal device 10 from the three-dimensional coordinates (xp1, yp1, zp1) of point P1 relative to the mobile terminal device 10 to calculate the three-dimensional coordinates (xr1, yr1, zr1) of the subtraction point R1. That is, the conversion control unit 105 changes the reference point of the three-dimensional coordinates (xp1, yp1, zp1) of point P1 of the point cloud P from the point cloud reference position P00 of the mobile terminal device 10 to the reference point of the geographic coordinate system, thereby converting the point cloud coordinate system of the point cloud P into the geographic coordinate system. This makes it possible to handle the three-dimensional coordinates of the point cloud as three-dimensional coordinates (measurement data) in the geographic coordinate system.

[0043] Here, the conversion control unit 105 may preliminarily calculate the three-dimensional difference D (dx, dy, dz) between the point cloud reference position P0 of the 3D laser scanner 11 and the position reference position R0 of the position communication device 13 (the point at which the current position of the mobile terminal device 10 is acquired), and use this three-dimensional difference D (dx, dy, dz) to subtract the three-dimensional difference D (dx, dy, dz) and the three-dimensional coordinates (xr0, yr0, zr0) of the current position R0 of the mobile terminal device 10 from the three-dimensional coordinates (xp1, yp1, zp1) of point P1 of the point cloud P0, thereby calculating the three-dimensional coordinates (xr1, yr1, zr1) of the subtraction point R1. This makes it possible to convert the point cloud coordinate system of the point cloud P into a geographic coordinate system with high precision, thereby improving the accuracy of creating two-dimensional drawings.

[0044] Here, when the position communication device 13 is a GPS receiver (single-frequency GNSS receiver) attached to the mobile terminal device 10, the accuracy (measurement limit) of the three-dimensional coordinates (xr0, yr0, zr0) of the current position R0 of the mobile terminal device 10 acquired by the conversion control unit 105 is generally about 10 m. Therefore, when an RTK receiver (not shown) is attached to the mobile terminal device 10 and the position communication device 13 is a two-frequency multi-GNSS receiver that is an RTK receiver, the accuracy of the three-dimensional coordinates (xr0, yr0, zr0) of the current point R0 of the mobile terminal device 10 acquired by the conversion control unit 105 is about several centimeters, and the accuracy of the three-dimensional coordinates (xr0, yr0, zr0) of the current point R0 of the mobile terminal device 10 is significantly improved. In this case, the accuracy of the three-dimensional coordinates (xr0, yr0, zr0) of the current position R0 of the mobile terminal device 10, which associates the point cloud P0 of the object O with the captured image I0, is also improved, thereby improving the accuracy of creating two-dimensional drawings.

[0045] Here, the three-dimensional coordinates (xr0, yr0, zr0) (GNSS information) of the current position R0 of the mobile terminal device 10 in the geographic coordinate system obtained by the dual-frequency multi-GNSS receiver may be obtained by a receiver built into the mobile terminal device 10 (smartphone) with a touch panel or the tablet-type terminal device 10, or may be obtained by a dual-frequency multi-GNSS receiver that is separately attached to the mobile terminal device 10 as described above.

[0046] In the above description, when the mobile terminal device 10 is in an environment where it can connect to a network on-site, the conversion control unit 105 acquires the three-dimensional coordinates (xr0, yr0, zr0) of the current position R0 of the mobile terminal device 10 in the geographic coordinate system on-site, but this is not limiting. For example, when the mobile terminal device 10 is in an environment where it cannot connect to a network on-site, the conversion control unit 105 cannot acquire the three-dimensional coordinates (xr0, yr0, zr0) of the current position R0 of the mobile terminal device 10 in the geographic coordinate system on-site using the position communication device 13 of the mobile terminal device 10. Therefore, for example, the conversion control unit 105 may use the position communication device 13 of the mobile terminal device 10 to acquire offline information related to positioning on-site. After that, when the mobile terminal device 10 is in an environment where it can connect to a network, the conversion control unit 105 may acquire, based on the offline information, three-dimensional coordinates (xr0, yr0, zr0) of the current position R0 of the mobile terminal device 10 in the geographic coordinate system at the time when the point cloud P and the captured image I were acquired. Examples of offline information related to positioning include observation data (RAW data) from a GNSS receiver and observation data from Geospatial Information Authority of Japan's electronic reference point. This method of acquiring position information (three-dimensional coordinates) is called static positioning (correction of position information). Static positioning is particularly effective when the mobile terminal device 10 is in an environment where it cannot connect to a network, such as deep in the mountains or in a forest. The conversion control unit 105 can obtain the three-dimensional coordinates (xr0, yr0, zr0) of the current position R0 of the mobile terminal device 10 in the geographic coordinate system through static positioning, and can convert the point cloud coordinate system of the point cloud P into the geographic coordinate system using the three-dimensional coordinates (xr0, yr0, zr0) of the current position R0.

[0047] In the above description, first, the three-dimensional coordinates of the point cloud P of the object O are obtained (FIG. 2: S102), then the captured image I in which the object O is captured is obtained (FIG. 2: S103), the point cloud coordinate system of the point cloud P is associated with the camera coordinate system of the captured image I (FIG. 2: S104), and finally, the point cloud coordinate system of the point cloud P is converted into a geographic coordinate system (FIG. 2: S105), but there is no particular limitation on the order in which these steps are performed. For example, first, the captured image I in which the object O is captured may be obtained (FIG. 2: S103), then the three-dimensional coordinates of the point cloud P of the object O may be obtained (FIG. 2: S102), the point cloud coordinate system of the point cloud P may be associated with the camera coordinate system of the captured image I (FIG. 2: S104), and finally, the point cloud coordinate system of the point cloud P may be converted into a geographic coordinate system (FIG. 2: S105). Furthermore, after associating the point cloud coordinate system of point cloud P with the camera coordinate system of captured image I (FIG. 2: S104), the point cloud coordinate system of point cloud P is converted into a geographic coordinate system (FIG. 2: S105). However, for example, the point cloud coordinate system of point cloud P may be converted into a geographic coordinate system (FIG. 2: S105) and then associated with the geographic coordinate system of point cloud P with the camera coordinate system of captured image I (FIG. 2: S104). This is because the three-dimensional coordinates of point cloud P are convertible between the point cloud coordinate system and the geographic coordinate system, and also the three-dimensional coordinates of the point cloud are convertible between the point cloud coordinate system or the geographic coordinate system and the camera coordinate system.

[0048] Regarding the three-dimensional coordinates of the point cloud P, the point cloud acquisition control unit 102 may convert the surface of the shape represented by the acquired three-dimensional coordinates of the point cloud P into mesh data representing the surface of the object O, which is composed of polygons, as shown in FIG. 5A. The point cloud acquisition control unit 102 converts the three-dimensional coordinates of the point cloud P into mesh data M, in which all faces are composed of triangles with three vertices, based on a meshing algorithm such as triangulation. Here, the three-dimensional coordinates of the point cloud P are composed of the items of a three-dimensional coordinate ID, an image ID, and the three-dimensional coordinates. Meanwhile, the mesh data M is data that expresses the three-dimensional coordinates of the point cloud P as a mesh (mesh), and is composed of the items of a mesh ID, a mesh vertex, and a normal vector. A predetermined number (one or more) of vertices of the three-dimensional coordinates are stored as the mesh vertices. A normal vector is a vector perpendicular to the mesh surface and perpendicular to all straight lines on the mesh surface, and a three-dimensional representation of the normal vector of the mesh surface is stored. Here, the mesh data M contains the vertices of the polygons that make up the mesh and the outward normal vectors of the polygons, so the amount of data is compressed compared to the three-dimensional coordinates of the individual point clouds P. In this way, by meshing the three-dimensional coordinates of the point clouds P, the point cloud data, which has a large amount of data, can be converted into mesh data M, which has a compressed amount of data, and can be easily handled. Furthermore, the mesh data M represents the surface of the object O, which makes it easy to paste an object image (described later), and therefore makes it easy for the surveyor to specify drawing points. Note that, although a triangle is used as the mesh shape in the above description, the type of mesh shape is not particularly limited, and in addition to a triangle, other shapes such as a rectangle and a pentagon can be used.

[0049] When the conversion control unit 105 completes the conversion, the acquisition of the captured image I and point cloud P from a specific viewpoint of the object O is completed. Next, the surveyor scans the object O with the 3D laser scanner 11 and camera 12. The point cloud acquisition control unit 102 acquires the three-dimensional coordinates of the point cloud P of the object O from different viewpoints (directions) with respect to the object O (FIG. 2: S102), the image acquisition control unit 103 acquires the captured image I (FIG. 2: S103), the correspondence control unit 104 associates the geographic coordinate system of the point cloud P with the camera coordinate system of the captured image I (FIG. 2: S104), and the conversion control unit 105 converts the point cloud coordinate system of the point cloud P into a geographic coordinate system (FIG. 2: S105). By repeating this process, multiple captured images I and point clouds P from different viewpoints (directions) with respect to the object O are acquired.

[0050] Now, when the surveyor inputs a predetermined display key into the mobile terminal device 10 to check the acquired point cloud P, the display designation control unit 106 of the mobile terminal device 10 displays the acquired point cloud P on a predetermined point cloud screen, and also displays any point R within the acquired point cloud P on the point cloud screen so that it can be specified (Figure 2: S106).

[0051] Here, there is no particular limitation on the display method of the display specification control unit 106, but for example, by associating the point cloud P from the point cloud coordinate system or the geographic coordinate system to the camera coordinate system, it is possible to display the point cloud P in the captured image I, and therefore, as shown in Fig. 5B, the display specification control unit 106 displays the point cloud P in the camera coordinate system as a point cloud screen S0 on a display reception unit (for example, a touch panel), and also displays an arbitrary point R1 of the point cloud P displayed on the touch panel so that it can be specified. This allows the surveyor to specify an arbitrary point R1 of the point cloud P while looking at the point cloud screen S0.

[0052] Here, if the number of data points in the point cloud P is large, the contours and detailed parts of the object O become clear, and it is possible that the surveyor will be able to accurately specify the desired part of the object O. However, if the number of data points in the point cloud P is small, it will be difficult for the surveyor to determine at a glance which points correspond to which parts of the object O, as shown in Figure 5B.

[0053] Therefore, when the surveyor inputs a predetermined switching key into the mobile terminal device 10, the switching control unit 107 of the mobile terminal device 10 pastes an object image IO showing the object in the captured image I onto the point cloud P based on multiple captured images I taken from different viewpoints (directions) of the object O, and switches to a predetermined image screen S1 that displays any point R in the point cloud P so that it can be specified via the object image IO (Figure 2: S107).

[0054] Here, there is no particular limitation on the switching method of the switching control unit 107. For example, when a surveyor appropriately moves the mobile terminal device 10 relative to the object O and acquires multiple captured images I of the object O from different viewpoints, the multiple captured images (here, two captured images I0 and I1) become multi-view images of the object O, as shown in FIG. 6A. Therefore, the switching control unit 107 reconstructs the position and orientation of the camera 12 and the three-dimensional structure 3DS of the object O from the multiple captured images I0 and I1. Examples of this reconstruction method include SfM (Structure from Motion) and MVS (Multi View Stereo).

[0055] Next, as shown in Fig. 6B, the switching control unit 107 matches the restored three-dimensional structure 3DS of the object O with the point group P of the object O, thereby associating the three-dimensional structure 3DS of the object O with the point group P. Here, the matching method is not particularly limited, and examples include methods of matching the position and orientation of the camera 12 in the restored three-dimensional structure 3DS of the object O with the position and orientation of the three-dimensional laser scanner 11 in the point group P of the object O, or matching feature points in the three-dimensional structure 3DS of the object O with feature points in the point group P of the object O.

[0056] Then, as shown in FIG. 7A, the switching control unit 107 pastes an object image IO (texture) that corresponds to the three-dimensional structure 3DS of the object O associated with the point cloud P and indicates the object O in the multiple captured images 10 and 11 onto the point cloud P, and displays an image screen S1 showing the point cloud P onto which the object image IO has been pasted on the touch panel as shown in FIG. 7B. Here, an arbitrary point R1 of the point cloud P within the image screen S1 is displayed so that it can be specified. In this way, by displaying the object image IO corresponding to the point cloud P on the image screen S1, even if the number of data points in the point cloud P is small, the surveyor can accurately specify a desired position within the object O while viewing the object image IO, thereby specifying point R1 within the point cloud P onto which the object image IO has been pasted.

[0057] 7B, the object image IO is cut by the contour of the object O, so the point group P can be seen between the object images IO, but here, the point group P is highlighted to make it easier to understand the present invention. In reality, the point group P is made up of extremely small points, so when the object image IO is attached to the point group P, the point group P becomes difficult to see.

[0058] Now, when the switching control unit 107 completes the switching, the surveyor selects a predetermined figure (e.g., a line segment) from among multiple types of figures on the point cloud screen S0 or the image screen S1, and specifies a predetermined point for the selected figure as a figure point.The drawing control unit 108 of the mobile terminal device 10 then obtains the three-dimensional coordinates of the specified figure point, and draws the figure based on the three-dimensional coordinates of the figure point (Figure 2: S108).

[0059] Here, the drawing method of the drawing control unit 108 is not particularly limited. For example, as shown in FIG. 8, a surveyor selects a line segment, checks a predetermined object image IO (e.g., a door) on the image screen S1 via the touch panel of the mobile terminal device 10, and specifies a predetermined point L1 for the line segment at a position corresponding to one end of the lower part of the door as the start point of the line segment. The drawing control unit 108 then acquires the two-dimensional coordinates (vl1, ul1) of the start point L1 of the line segment in the camera coordinate system of the image screen S1 and converts them into three-dimensional coordinates (xn1, yn1, zn1) of a corresponding point N1 in the geographic coordinate system using the above-mentioned conversion information. The drawing control unit 108 then determines whether point R of the point cloud P exists within a predetermined specified range (a circular or square range) centered on the three-dimensional coordinates (xn1, yn1, zn1) of the converted corresponding point N1.

[0060] As a result of the determination, point R1 of point cloud P exists within the predetermined specified range, so the drawing control unit 108 acquires the three-dimensional coordinates (xr1, yr1, zr1) of the point R1 of point cloud P that exists. This allows the surveyor to accurately specify the start point L1 for the line segment while looking at the image screen S1. Here, if multiple points R of point cloud P exist within the specified range, the drawing control unit 108 only needs to acquire the three-dimensional coordinates (xr1, yr1, zr1) of the point R1 that is closest to the corresponding point N1 among the multiple points R that exist.

[0061] Furthermore, as shown in Figure 8, if the surveyor can look at the point cloud screen S0 and recognize one end of the lower part of the door, he or she can simply specify the starting point L1 of the line segment at the position corresponding to this end of the lower part of the door.

[0062] 9, the surveyor specifies a predetermined point L2 for the line segment as the end point of the line segment at a position on the image screen S1 corresponding to the other end of the lower part of the door in the object image IO. The rendering control unit 108 then renders the line segment L on the image screen S1 based on the start point L1 and end point L2 of the line segment, obtains the two-dimensional coordinates (vl2, ul2) of the end point L2 of the line segment in the camera coordinate system, and converts them into three-dimensional coordinates (xn2, yn2, zn2) of the corresponding point N2 in the geographic coordinate system using the conversion information. The rendering control unit 108 then determines whether point R of the point group P exists within a predetermined specified range centered on the three-dimensional coordinates (xn2, yn2, zn2) of the converted corresponding point N2.

[0063] As a result of the determination, point R2 of point group P is also present within the predetermined specified range, so the drawing control unit 108 acquires the three-dimensional coordinates (xr2, yr2, zr2) of point R2 of point group P. This enables the surveyor to draw line segment L within the space of the geographic coordinate system based on the three-dimensional coordinates (xr1, yr1, zr1) of the start point L1 of line segment L and the three-dimensional coordinates (xr2, yr2, zr2) of the end point L2.

[0064] As described above, if the surveyor can recognize the other lower end of the door by looking at the point cloud screen S0, as shown in Figure 9, he or she can specify the end point L2 of the line segment at a position corresponding to the lower end of this door, as described above.

[0065] In S108, if there is no point R in the point cloud P corresponding to the figure point designated by the surveyor, the following occurs. That is, as shown in FIG. 10, for example, suppose that the surveyor designates a starting point L1 for a line segment, and then designates a predetermined point L2 for the line segment as an end point of the line segment via the touch panel of the mobile terminal device 10 at a position on the image screen S1 where point R of the point cloud P does not exist. Then, the rendering control unit 108 acquires the two-dimensional coordinates (vl2, ul2) of the end point L2 of the line segment in the camera coordinate system and converts them into three-dimensional coordinates (xn2, yn2, zn2) of the corresponding point N2 in the geographic coordinate system using the conversion information. The rendering control unit 108 then determines whether point R of the point cloud P exists within a predetermined specified range centered on the three-dimensional coordinates (xn2, yn2, zn2) of the converted corresponding point N2.

[0066] As a result of the determination, point R of point cloud P does not exist within the predetermined specified range, so the rendering control unit 108 acquires the three-dimensional coordinates (xn2, yn2, zn2) of corresponding point N2 corresponding to end point L2 of the specified line segment. As a result, even if point R of point cloud P does not exist, by converting the two-dimensional coordinates of figure point L2 in the camera coordinate system into the three-dimensional coordinates of corresponding point N2 in the geographic coordinate system, it becomes possible to freely render a figure in the space of the geographic coordinate system. This is particularly effective when, for example, it is desired to render the outline or shape of an object that is planned to be constructed in the future from the current object O.

[0067] As described above, the same result occurs even if the surveyor looks at the point cloud screen S0 and specifies the end point L2 of the line segment at a position where point R of the point cloud P does not exist, as shown in FIG.

[0068] Now, after the drawing control unit 108 has completed drawing and the line segment drawing has been drawn, the surveyor inputs a specified two-dimensional drawing creation key, and the creation control unit 109 of the mobile terminal device 10 creates a two-dimensional drawing in the geographic coordinate system based on the three-dimensional coordinates of the specified figure points (Figure 2: S109).

[0069] Here, the creation method of the creation control unit 109 is not particularly limited, but for example, if a surveyor inputs a key to create a two-dimensional plan view in which the horizontal direction (left-right direction) of the geographic coordinate system is the x-axis direction and the depth direction (front-back direction) of the geographic coordinate system is the z-axis direction, the creation control unit 109 acquires the two-dimensional coordinates (xr1, zr1) in the x-axis and z-axis directions from the three-dimensional coordinates (xr1, yr1, zr1) of the start point L1 of the line segment L, as shown in Figure 11, and plots the two-dimensional coordinates (xr1, zr1) of the start point L1 on the plan view. Also, the creation control unit 109 acquires the two-dimensional coordinates (xr2, zr2) in the x-axis and z-axis directions from the three-dimensional coordinates (xr2, yr2, zr2) of the end point L2 of the line segment L, and plots the two-dimensional coordinates (xr2, zr2) of the end point L2 on the plan view. The creation control unit 109 then creates a line segment L connecting the start point L1 and the end point L2 on the floor plan, and displays the two-dimensional floor plan. This allows the surveyor to easily obtain a two-dimensional floor plan. In particular, because the drawn line segment L is based on the geographic coordinate system, it can be directly combined with two-dimensional drawings (CAD drawings, DXF drawings, etc.) used in other fields, improving work efficiency.

[0070] Furthermore, if the surveyor inputs a key to create a two-dimensional front view in which the horizontal direction (left-right direction) of the geographic coordinate system is the x-axis direction and the vertical direction (up-down direction) of the geographic coordinate system is the y-axis direction, the creation control unit 109 acquires the two-dimensional coordinates (xr1, yr1) in the x-axis and y-axis directions from the three-dimensional coordinates (xr1, yr1, zr1) of the start point L1 of the line segment L, as shown in FIG. 11, and plots the two-dimensional coordinates (xr1, yr1) of the start point L1 on the front view. Also, the creation control unit 109 acquires the two-dimensional coordinates (xr2, yr2) in the x-axis and y-axis directions from the three-dimensional coordinates (xr2, yr2, zr2) of the end point L2 of the line segment L, and plots the two-dimensional coordinates (xr2, yr2) of the end point L2 on the front view. The creation control unit 109 then creates a line segment L connecting the start point L1 and the end point L2 on the front view, and displays the two-dimensional front view. This allows other two-dimensional drawings to be easily created. Furthermore, the created two-dimensional drawing is saved in association with the predetermined name 303a ("AAA") that was initially entered, so the surveyor can quickly call up the two-dimensional drawing using the name 303a.

[0071] In the above, we have used two-dimensional plan views in the x-axis and z-axis directions and two-dimensional front views in the x-axis and y-axis directions as examples, but this is not limited to these, and the same applies to, for example, two-dimensional side views or cross-sectional views in the y-axis and z-axis directions.

[0072] Furthermore, in the above description, the creation control unit 109 creates a two-dimensional drawing using the three-dimensional coordinates of point P in point group P0, but as described above, the two-dimensional drawing may also be created using the three-dimensional coordinates of corresponding point N. For example, as described above, if the start point L1 of line segment L is the three-dimensional coordinates (xr1, yr1, zr1) of point R1 in point group P and the end point L2 of line segment L is the three-dimensional coordinates (xn2, yn2, zn2) of corresponding point N2, when a key for creating a two-dimensional floor plan is input, as shown in FIG. 13, the creation control unit 109 plots the two-dimensional coordinates (xr1, zr1) of start point L1 in the x-axis direction and the z-axis direction and the two-dimensional coordinates (xn2, zn2) of end point L2 in the x-axis direction and the z-axis direction on the floor plan, creates a line segment L connecting start point L1 and end point L2 on the floor plan, and displays the two-dimensional floor plan. Furthermore, when a key for creating a two-dimensional front view is input, the creation control unit 109 plots the two-dimensional coordinates (xr1, yr1) of the start point L1 in the x-axis direction and the y-axis direction and the two-dimensional coordinates (xn2, yn2) of the end point L2 in the x-axis direction on the front view, creates a line segment L connecting the start point L1 and the end point L2 on the front view, and displays a two-dimensional plan view. This allows the surveyor to freely draw figures and create two-dimensional drawings.

[0073] While the above description employs line segments as the shape, other types of shapes may be used. For example, as shown in FIG. 13A , if the shape is a rectangle, the center point of the rectangle can be designated as a shape point and the length and width of the rectangle can be input. The rendering control unit 108 can then draw the rectangle by expanding the length and width from the three-dimensional coordinates of the designated center point. Here, point R of point cloud P is based on a geographic coordinate system, so the length and width can be input in actual dimensions, thereby drawing a rectangle that reflects real-world dimensions. Alternatively, if the shape is a rectangle, the four corner points of the rectangle can be designated as shape points, and the rendering control unit 108 can then draw the rectangle based on the three-dimensional coordinates of the designated four corner points. This also applies to pentagons and polygons. For example, if the figure is a circle, by specifying the center point of the circle as a figure point and inputting the radius, the drawing control unit 108 can draw the circle by expanding the radius from the three-dimensional coordinates of the specified center point. Furthermore, for example, if the figure is a broken line, by specifying the bend point of the broken line as a figure point, the drawing control unit 108 can draw the broken line from the three-dimensional coordinates of the specified bend point.

[0074] Here, if the figure is a rectangle, for example, as shown in FIG. 13B, if a surveyor wants to create a two-dimensional drawing of the outline of a door on the image screen S1, he or she can select the rectangle and specify the four corners of the door as figure points, thereby drawing a rectangle connecting the four corners of the door from the point cloud P. Here, by using the three-dimensional coordinates of the four corners of the door, a two-dimensional drawing of the outline of the door can be easily created. The object O is not limited to a door, and can be applied to any building or construction object in various fields such as construction, civil engineering, forestry, surveying, and disaster prevention.

[0075] An embodiment of the present invention will now be described. First, the drawing creation system 1 and drawing creation method shown in FIGS. 1 to 13 were embodied in a mobile terminal device 10, and figures were drawn within a point cloud and a two-dimensional drawing was created. First, assuming the interior of a single-family home as the object O, a surveyor launches a predetermined application on the mobile terminal device 10 and selects the start key. Then, the surveyor points the three-dimensional laser scanner 11 and camera 12 of the mobile terminal device 10 toward the object O to acquire a point cloud P and a captured image I of the object O.

[0076] 14A, a captured image I and a point cloud P are displayed on the touch panel of the mobile terminal device 10. The point cloud P is displayed in a square portion W (small window) provided in the upper right corner of the captured image I. The mobile terminal device 10 associates the geographic coordinate system of the point cloud P with the camera coordinate system of the captured image I, and converts the point cloud coordinate system of the point cloud P into the geographic coordinate system. Note that mesh data M converted from the point cloud P is faintly displayed superimposed on the captured image I.

[0077] Next, the surveyor scans the object O with the three-dimensional laser scanner 11 and the camera 12, and the mobile terminal device 10 acquires multiple captured images I and a point cloud P of the object O from different viewpoints (directions).

[0078] Here, when the surveyor inputs a predetermined display key into the mobile terminal device 10, as shown in Fig. 14B, the mobile terminal device 10 displays a point cloud screen S0 on which any point within the acquired point cloud P can be specified. Here, as shown in Fig. 14B, if the number of data points in the point cloud P is small, it is difficult to see at a glance which point corresponds to which part of the object O on the point cloud screen S0.

[0079] Therefore, when the surveyor inputs a predetermined switching key into the mobile terminal device 10, as shown in Fig. 14B, the mobile terminal device 10 displays an image screen S1 in which the object image IO is pasted onto the point cloud P. This makes it clear that although it was difficult to identify the object O in the point cloud P, by pasting the object image IO, the object O can be clearly identified.

[0080] When the surveyor selects a line segment as a predetermined figure while viewing the image screen S1 and specifies the center point of the tatami mat of the object O as the starting point, the mobile terminal device 10 acquires the three-dimensional coordinates of the specified starting point L1 and displays the starting point L1 on the image screen S1, as shown in Fig. 15A. The surveyor can specify a point of the point cloud P from the object image IO on the image screen S1, which allows the surveyor to specify the starting point L1 at a desired position with high precision. For example, because the boundaries and four corners of the tatami mat of the object O are clearly shown on the object image IO, the surveyor can accurately specify the boundaries and four corners of the tatami mat of the object O.

[0081] Now, when a surveyor specifies an end point L2 of a line segment while viewing the point cloud screen S0, for example, the mobile terminal device 10 acquires the three-dimensional coordinates of the specified end point L2 and displays the end point L2 on the point cloud screen S0, as shown in FIG. 15B. The mobile terminal device 10 also draws a line segment L based on the three-dimensional coordinates of the start point L1 and the end point L2. This allows the surveyor to easily draw the line segment L. Here, the length of the line segment L ("1.12 m") is calculated and displayed based on the three-dimensional coordinates of the start point L1 and the end point L2. In this way, by utilizing the three-dimensional coordinates of the start point L1 and the end point L2, the actual dimensions can also be easily calculated.

[0082] Here, the surveyor can easily draw the broken line L within the point cloud P of the point cloud image S0 by selecting, for example, a broken line L as a figure and specifying multiple figure points, as shown in Figure 15B. Also, the length of each figure point of the broken line L is calculated and displayed based on the three-dimensional coordinates of each figure point. Furthermore, the direction in which the point cloud P is displayed in the point cloud image S0 can be freely set.

[0083] Furthermore, when the surveyor selects, for example, a rectangle L as a figure, inputs the length and width ("1.00 m"), and specifies a figure point at the location where the rectangle L is to be placed, the mobile terminal device 10 acquires the three-dimensional coordinates of the specified figure point and draws the rectangle L from the length and width, as shown in FIG. 16A. The length and width are displayed on the rectangle L. Furthermore, when the rectangle L is selected, the four corner points and center point that make up the rectangle L are displayed. The rectangle L can be moved by selecting and moving these points.

[0084] Furthermore, when the surveyor selects a line segment L as the figure and specifies the start point and end point, the mobile terminal device 10 acquires the three-dimensional coordinates of the specified start point and end point, and draws the line segment L in addition to the rectangle L, as shown in Fig. 16B. Also, when the surveyor selects a circle L as the figure, inputs a radius ("0.50 m"), and specifies a figure point at the position where the circle L is to be placed, the mobile terminal device 10 acquires the three-dimensional coordinates of the specified figure point, and draws the circle L from the radius, as shown in Fig. 16B. In this way, the surveyor can draw various figures in the point cloud P.

[0085] Then, when the surveyor inputs a key for creating a plan view from among the two-dimensional drawings, the mobile terminal device 10 creates a two-dimensional plan view A in the geographic coordinate system based on the three-dimensional coordinates of the specified figure points, as shown in Fig. 16B. Here, a grid representing the two-dimensional plan view is displayed in the geographic coordinate system. In this way, it can be seen that the surveyor can easily create the two-dimensional drawing A.

[0086] Here, because the three-dimensional coordinates of the figure points are based on the geographic coordinate system, when linked to an external map database, for example, it is possible to display two-dimensional plan view A and a nearby map B from the map database, which has three-dimensional coordinates close to the three-dimensional coordinates of the figure drawn on the two-dimensional plan view A, superimposed on each other, as shown in Figure 17. This makes it possible to combine the created two-dimensional plan view A with an existing map database and smoothly proceed with the design, planning, and changes of buildings and constructions.

[0087] In the embodiment of the present invention, the mobile terminal device 10 is configured to include each control unit. However, the program that realizes each unit may be stored in a storage medium and the storage medium may be provided. In this configuration, the program is read into the device, and the device realizes each control unit. In this case, the program itself read from the recording medium achieves the effects of the present invention. Furthermore, it is also possible to provide a method for storing the steps executed by each control unit on a hard disk. [Industrial Applicability]

[0088] As described above, the drawing creation system and drawing creation method of the present invention are useful in all fields that utilize two-dimensional drawings in a geographic coordinate system, such as construction, civil engineering, forestry, surveying, and disaster prevention, and are effective as a drawing creation system and drawing creation method that can accurately draw figures within a point cloud and easily create two-dimensional drawings. [Explanation of symbols]

[0089] 1. Drawing creation system 10. Portable terminal device 11 3D laser scanner 12 Camera 13 Location communication device 101 Selection reception control unit 102 Point cloud acquisition control unit 103 Image acquisition control unit 104 Response control section 105 Conversion control unit 106 Display specification control unit 107 Switching control unit 108 Drawing control unit 109 Creation control section

Claims

1. A drawing creation system equipped with a mobile terminal device in which a three-dimensional laser scanner and a camera are installed facing the same direction, a point cloud acquisition control unit that acquires three-dimensional coordinates of a point cloud of the object in a point cloud coordinate system by scanning the object with the three-dimensional laser scanner; an image acquisition control unit that acquires a captured image of the object by photographing the object with the camera; a correspondence control unit that associates a point cloud coordinate system of the point cloud with a camera coordinate system of the captured image; a conversion control unit that acquires three-dimensional coordinates of a current position of the mobile terminal device in a geographic coordinate system using a position communication device attached to the mobile terminal device, and converts the point cloud coordinate system of the point cloud into the geographic coordinate system; a display designation control unit that displays the acquired point cloud on a predetermined point cloud screen and displays any point in the acquired point cloud on the point cloud screen so that the point cloud can be designated; a switching control unit that, when a predetermined switching key is input, switches to a predetermined image screen that displays an object image showing the object in the captured images on the point cloud based on a plurality of captured images of the object from different viewpoints, and allows any point in the point cloud to be specified via the object image; a drawing control unit that, when a predetermined point for a figure is designated as a figure point on the point cloud screen or the image screen, acquires three-dimensional coordinates of the figure point and draws the figure based on the three-dimensional coordinates of the figure point; a creation control unit that creates a two-dimensional drawing in a geographic coordinate system based on the three-dimensional coordinates of the figure points when a predetermined two-dimensional drawing creation key is input after the figure has been drawn; A drawing creation system comprising:

2. the switching control unit restores the position and attitude of the camera and the three-dimensional structure of the object from the plurality of captured images, matches the restored three-dimensional structure of the object with a point cloud of the object, associates the three-dimensional structure of the object with the point cloud, and displays the image screen by corresponding to the three-dimensional structure of the object associated with the point cloud and pasting an object image showing the object in the plurality of captured images onto the point cloud; The drawing creation system according to claim 1 .

3. When the graphic point is designated, the drawing control unit acquires two-dimensional coordinates of the graphic point in a camera coordinate system of the point cloud screen or the image screen, converts the two-dimensional coordinates into three-dimensional coordinates of a corresponding point in a geographic coordinate system using predetermined conversion information, and determines whether or not a point of the point cloud exists within a predetermined specified range centered on the three-dimensional coordinates of the converted corresponding point; If a point of the point cloud exists within the specified range as a result of the determination, the rendering control unit acquires three-dimensional coordinates of the point of the point cloud that exists within the specified range, If the result of the determination is that no point of the point cloud exists within the specified range, the rendering control unit acquires three-dimensional coordinates of the corresponding point. The drawing creation system according to claim 1 .

4. A drawing creation method for a drawing creation system equipped with a mobile terminal device in which a three-dimensional laser scanner and a camera are installed facing the same direction, a point cloud acquisition control step of acquiring three-dimensional coordinates of a point cloud of the object in a point cloud coordinate system by scanning the object with the three-dimensional laser scanner; an image acquisition control step of acquiring a captured image of the object by photographing the object with the camera; a correspondence control step of associating a point cloud coordinate system of the point cloud with a camera coordinate system of the captured image; a conversion control step of acquiring three-dimensional coordinates of a current position of the mobile terminal device in a geographic coordinate system using a position communication device attached to the mobile terminal device, and converting the point cloud coordinate system of the point cloud into the geographic coordinate system; a display designation control step of displaying the acquired point cloud on a predetermined point cloud screen and displaying any point in the acquired point cloud on the point cloud screen in a designable manner; a switching control step of, when a predetermined switching key is input, pasting an object image showing the object in the photographed images onto the point cloud based on a plurality of photographed images of the object from different viewpoints, and switching to a predetermined image screen that displays an arbitrary point in the point cloud so that the arbitrary point can be specified via the object image; a drawing control step of, when a predetermined point for a figure is designated as a figure point on the point cloud screen or the image screen, acquiring three-dimensional coordinates of the figure point and drawing the figure based on the three-dimensional coordinates of the figure point; a creation control step of creating a two-dimensional drawing in a geographic coordinate system based on the three-dimensional coordinates of the figure points when a predetermined two-dimensional drawing creation key is input after the figure has been drawn; A drawing creation method comprising:

Images