Point cloud processing device, point cloud processing method, program, and point cloud processing system

Abstract

To enable appropriate alignment of multiple point clouds.SOLUTION: A point cloud processing device comprises display screen generation means for generating a display screen obtained by superimposing a plurality of point clouds representing a three-dimensional point cloud acquired by measuring a target object on an image representing the target object.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a point cloud processing device, a point cloud processing method, a program, and a point cloud processing system.

Background Art

[0002] Patent Document 1 describes a point cloud data processing device that improves the efficiency of the work of aligning a plurality of point cloud data by including a highlighting control unit that highlights the markers of the corresponding other point cloud when a marker in one point cloud is specified in a state where a first point cloud including a plurality of markers for alignment and a second point cloud including a plurality of markers for alignment are displayed.

Summary of the Invention

Problems to be Solved by the Invention

[0003] An object of the present invention is to appropriately align a plurality of point clouds.

Means for Solving the Problems

[0004] The point cloud processing device according to the present invention includes display screen generation means for generating a display screen in which a plurality of point clouds indicating three-dimensional point clouds acquired by measuring an object are superimposed on an image indicating the object.

Effects of the Invention

[0005] According to the present invention, a plurality of point clouds can be appropriately aligned.

Brief Description of the Drawings

[0006]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

MODE FOR CARRYING OUT THE INVENTION

[0007] In industries such as civil engineering and architecture, BIM (Building Information Modeling) / CIM (Construction Information Modeling) is being promoted for the purpose of addressing issues such as the declining birthrate and aging population and improving labor productivity.

[0008] BIM is a solution for information utilization in all processes from the design, construction to the maintenance and management of buildings, which is a database of buildings that adds attribute data such as cost, finish, and management information to a three-dimensional digital model of a building created on a computer (hereinafter referred to as a 3D model).

[0009] CIM is a solution for the civil engineering field (for all infrastructure such as roads, electricity, gas, water supply, etc.) proposed following BIM that has been promoted in the construction field. Similar to BIM, it is being worked on to improve and sophisticate a series of construction production systems by sharing information among related parties centered around a 3D model.

[0010] What is important in promoting BIM / CIM is how to easily obtain 3D information on objects such as the spaces of buildings and public facilities. The 3D information mentioned here refers to a three-dimensional point cloud (hereinafter sometimes simply referred to as a point cloud) that holds distance information to the object, obtained by measuring the object with a laser scanner (hereinafter referred to as LS), etc., a mesh object generated based on the point cloud data indicating the three-dimensional point cloud, a 3D-CAD (Computer Aided Design) model, and the like.

[0011] When constructing a structure from scratch, since a complete product can be designed from scratch using BIM / CIM software, etc., BIM / CIM implementation is easy to introduce. On the other hand, in the case of existing buildings, there are cases where the design drawings at the time of construction do not remain, or the drawings at the time of design are different from the current situation due to renovations over time, etc., and the hurdle for BIM / CIM implementation increases. The BIM implementation of such existing buildings is called As-Build BIM, etc., and it has become an important issue for promoting future BIM / CIM implementation.

[0012] As a means of realizing As-Build BIM, there is a workflow of using the above-described LS for spatial measurement and creating a 3D-CAD model from the measured point cloud data. Conventionally, methods such as measurement using photos and measuring instruments, or methods of sketching by hand have been used for this work. However, due to the size of the space, the presence or absence of installed objects, and complexity (such as the way pipes are intertwined), a great deal of work cost may be incurred. Therefore, introducing LS that can acquire 3D information of the space has been attracting attention as an effective method for solving this problem.

[0013] In As-Build using LS, although the acquisition of 3D information has become easier, a new task of point cloud processing for point cloud data that did not exist in conventional work has emerged. In general point cloud processing, "multi-point measurement using LS", "generation of an integrated point cloud by aligning each point cloud", "removal of unnecessary point clouds such as noise", "mesh conversion of the point cloud, texture mapping to the mesh, 3D-CAD model conversion", "alignment processing", etc. are carried out.

[0014] In the alignment process of aligning the first point cloud with the second point cloud, if these point clouds are point clouds acquired (photographed) with a wall separating them, there is no common part between the two point clouds, so alignment cannot be properly performed. Also, in order to create a common part (overlap) between the point clouds, it was necessary to make a device for intentionally creating an overlap at the time of photographing, such as photographing in small pieces between one room and the other room, or opening the door to obtain the point cloud of the adjacent room.

[0015] In view of the above problems, a first object of the present invention is to properly align a plurality of point clouds even when there is no common part among the plurality of point clouds.

[0016] Another object is to reduce the burden on the user for post-processing when the alignment does not reach the desired quality after performing the alignment process.

[0017] (First Embodiment) FIG. 1 is an overall configuration diagram of a point cloud processing system according to a first embodiment of the present invention. The point cloud processing system 1 of the present embodiment is constructed by a terminal device 3 which is an example of a communication terminal and a management server 5.

[0018] FIG. 1 is an overall configuration diagram of a point cloud processing system according to an embodiment of the present invention. The point cloud processing system 1 of the present embodiment is constructed by a terminal device 3 which is an example of a communication terminal and a management server 5.

[0019] The management server 5 is an example of a point cloud processing device that executes one or more point cloud processes on point cloud data indicating a three-dimensional point cloud.

[0020] Here, the three-dimensional point cloud is an aggregate of coordinate points in the X, Y, Z directions, etc., corresponding to the measurement points on the surface of an object when measuring a certain space of the object using a laser scanner LS or the like. Each coordinate point is represented, for example, as (1, 3, 5). Also, color information may be added to each coordinate point, and the RGB values of each coordinate point may be added as color information. The three-dimensional point cloud is sometimes called a point cloud. The point cloud data is data that can be handled by a computer or the like as an aggregate of coordinate points in a virtual three-dimensional space.

[0021] In the above, an example of measuring a three-dimensional point cloud using a laser scanner LS has been shown, but the three-dimensional point cloud may be measured using other optical measurement means or mechanical measurement means. Examples of other optical measurement means include a method using a stereo camera and a method using Visual SLAM (Simultaneous Localization And Mapping).

[0022] The terminal device 3 and the management server 5 can communicate via the communication network 100. The communication network 100 is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), etc. The communication network 100 may include networks by not only wired communication but also wireless communication such as 3G (3rd Generation), WiMAX (Worldwide Interoperability for Microwave Access), LTE (Long Term Evolution), 5G (5th Generation), etc. Further, the terminal device 3 can communicate by a short-range communication technology such as NFC (Near Field Communication) (registered trademark).

[0023] <Hardware Configuration> FIG. 2 is a hardware configuration diagram of the terminal device and the management server according to this embodiment. Each hardware configuration of the terminal device 3 is indicated by a reference numeral in the 300s. Each hardware configuration of the management server 5 is indicated by a reference numeral in the 500s within parentheses.

[0024] The terminal device 3 includes a CPU (Central Processing Unit) 301, a ROM (Read Only Memory) 302, a RAM (Random Access Memory) 303, an HD (Hard Disk) 304, an HDD (Hard Disk Drive) 305, a recording medium 306, a media I / F 307, a display 308, a network I / F 309, a keyboard 311, a mouse 312, a CD-RW (Compact Disc-Re Writable) drive 314, and a bus line 310.

[0025] Among these, the CPU 301 controls the overall operation of the terminal device 3. The ROM 302 stores programs used to drive the CPU 301. The RAM 303 is used as a work area for the CPU 301. The HD 304 stores various data such as programs. The HDD 305 controls the reading or writing of various data to and from the HD 304 according to the control of the CPU 301. The media I / F 307 controls the reading or writing (storage) of data to and from a recording medium 306 such as a flash memory. The display 308 displays various information such as a cursor, menu, window, characters, or images. The network I / F 309 is an interface for data communication using the communication network 100. The keyboard 311 is a type of input means having a plurality of keys for inputting characters, numerical values, various instructions, etc. The mouse 312 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The CD-RW drive 314 controls the reading or writing of various data to and from a CD-RW 513 as an example of a removable recording medium. The terminal device 3 may further include a configuration for controlling the reading or writing (storage) of data to and from an external PC (Personal Computer) or an external device connected by wire or wirelessly such as Wi-Fi (Wireless Fidelity).

[0026] Also, the management server 5 includes a CPU 501, a ROM 502, a RAM 503, an HD 504, an HDD 505, a recording medium 506, a media I / F 507, a display 508, a network I / F 509, a keyboard 511, a mouse 512, a CD-RW drive 514, and a bus line 510. Since these have the same configuration as the above-described configuration (CPU 301, ROM 302, RAM 303, HD 304, HDD 305, recording medium 306, media I / F 307, display 308, network I / F 309, keyboard 311, mouse 312, CD-RW drive 314, and bus line 310), the description thereof is omitted.

[0027] Note that the CD-RW drive 314(514) may be replaced with a CD-R drive or the like. Also, the terminal device 3 and the management server 5 may each be constructed by a single computer, or may be constructed by a plurality of computers to which each part (function, means, or storage unit) is arbitrarily divided and assigned.

[0028] FIG. 3 is a functional block diagram of the point cloud processing system according to the present embodiment.

[0029] <Functional Configuration of Terminal Device> As shown in FIG. 3, the terminal device 3 includes a transmission / reception unit 31, a reception unit 32, a display control unit 34, and a storage / readout unit 39. Each of these units is a function or means realized by any of the components shown in FIG. 2 operating according to an instruction from the CPU 301 developed on the RAM 303 from the HD 304. Also, the terminal device 3 has a storage unit 3000 constructed by the RAM 303 and HD 304 shown in FIG. 2.

[0030] (Functional Configuration of Terminal Device) Next, each component of the terminal device 3 will be described.

[0031] The transmission / reception unit 31 is an example of reception means, and is realized by an instruction from the CPU 301 shown in FIG. 2 and the network I / F 309, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 100.

[0032] The reception unit 32 is an example of reception means, and is mainly realized by an instruction from the CPU 301 shown in FIG. 2 and the keyboard 311 and mouse 312, and receives various inputs from the user.

[0033] The display control unit 34 is an example of display control means, and is realized by an instruction from the CPU 301 shown in FIG. 2, and causes a display 308, which is an example of a display unit, to display various images and screens.

[0034] The memory / readout unit 39 is an example of memory control means, and performs processes of storing various data in the storage unit 3000, the recording medium 306, the CD-RW 313, and an external PC or external device, or reading out various data from the storage unit 3000, the recording medium 306, the CD-RW 313, and an external PC or external device, based on instructions from the CPU 301 shown in FIG. 2 and executed by the HDD 305, the media I / F 307, the CD-RW drive 314, and an external PC or external device.

[0035] <Functional configuration of the management server> The management server 5 includes a transmission / reception unit 51, a processing unit 53, a determination unit 55, a generation unit 57, and a memory / readout unit 59. Each of these units is a function or means that operates by the instructions from the CPU 501 according to a program expanded from the HD 504 onto the RAM 503 by any of the components shown in FIG. 2. Also, the management server 5 has a storage unit 5000 constructed by the HD 504 shown in FIG. 2. The storage unit 5000 is an example of storage means.

[0036] (Functional configurations of the management server) Next, each component of the management server 5 will be described. The management server 5 may be configured to distribute and implement each function across a plurality of computers. Furthermore, although the management server 5 is described as a server computer existing in a cloud environment, it may be a server existing in an on-premises environment.

[0037] The transmission / reception unit 51 is an example of transmission means, and performs transmission and reception of various data (or information) with other terminals, devices, or systems via the communication network 100, based on instructions from the CPU 501 shown in FIG. 2 and realized by the network I / F 509.

[0038] The processing unit 53 is realized by instructions from the CPU 501 shown in FIG. 2 and performs various processes including alignment processing. The processing unit 53 is an example of point cloud processing means.

[0039] The determination unit 55 is realized by an instruction from the CPU 501 shown in FIG. 2 and makes various determinations.

[0040] The generation unit 57 is realized by an instruction from the CPU 501 shown in FIG. 2 and performs various generations such as screen generation described later.

[0041] The storage / reading unit 59 is an example of storage control means and is executed by an instruction from the CPU 501 shown in FIG. 2, as well as by the HDD 505, the media I / F 507, the CD-RW drive 514, and an external PC or external device. It stores various data in the storage unit 5000, the recording medium 506, the CD-RW 513, and an external PC or external device, and reads out various data from the storage unit 5000, the recording medium 506, the CD-RW 513, and an external PC or external device. The storage unit 5000, the recording medium 506, the CD-RW 513, and an external PC or external device are examples of storage means.

[0042] In the storage unit 5000, a user information management DB 5001, a setting information management DB 5002, a storage process management DB 5003, a point cloud management DB 5004, and a process result management DB 5005 are constructed by a setting information management table.

[0043] The user information management DB 5001 stores and manages the file name of three-dimensional point cloud data in association with user information. The setting information management DB 5002 stores and manages various setting information. The storage process management DB 5003 stores and manages various processing programs for executing point cloud processing. The point cloud management DB 5004 stores and manages point cloud data. The process result management DB 5005 stores and manages process result information indicating the result of executing point cloud processing on the point cloud data.

[0044] <Functional configuration of the alignment processing unit> The processing unit 53 includes an alignment processing unit 530 that performs point cloud alignment processing. FIG. 4 is a functional block diagram of the alignment processing unit 530 according to the present embodiment. The alignment processing unit 530 includes a drawing analysis unit 531, a point cloud analysis unit 532, a pair selection unit 533, a position estimation unit 534, and a display image generation unit 535.

[0045] The drawing analysis unit 531 analyzes the drawing data to generate matching candidate images such as walls and columns in the drawing. The drawing data input to the drawing analysis unit 531 is image data obtained by imaging drawings such as floor plans of buildings and public facilities that are targets for acquiring point cloud data, and is provided in, for example, bitmap format or JPEG (Joint Photographic Experts Group) format. The drawing data may be image data obtained by capturing a paper drawing with a scanner or the like.

[0046] FIG. 5 is a diagram showing an example of the drawing data. The drawing data shown in FIG. 5 is a floor plan, where the walls and columns of the building are represented by straight lines, and the doors and windows are represented by predetermined symbols. Also, the walls are often represented as a combination of rectangles with thick sides.

[0047] FIG. 6 is a diagram showing an example of the matching candidate image. The drawing analysis unit 531 analyzes the drawing data using edge detection and straight line detection to generate a matching candidate image as shown in FIG. 6. Specifically, if a pixel detected as an edge is on a pixel detected as a straight line, that pixel becomes a matching candidate pixel. In FIG. 6, the white area indicates the pixels detected as matching candidates (candidate pixels), and the black area indicates the other pixels. In this way, in the generated matching candidate image, the surfaces of walls and columns are detected as matching candidate pixels. Note that, as shown in FIG. 6, the drawing analysis unit 531 discriminates the area with the symbol indicating the door in the floor plan and excludes it from the target of edge detection.

[0048] In this embodiment, it is assumed that the scale (scaling factor) of the drawing data is known, and based on this scale, the length on the matching candidate image is compared with the length of the point cloud for alignment processing. For example, if it is known that the length of 1000 pixels in the matching candidate image corresponds to 10 meters (m) on the drawing according to a known scale, the length of one pixel is set to 10 / 1000 = 0.01 m and compared with the point cloud. If the scale is not known, a provisional scale is set as in the embodiment described later, and the actual scale is estimated during the alignment processing.

[0049] The point cloud analysis unit 532 extracts matching candidate point clouds belonging to walls, pillars, etc. from the point cloud data obtained by measuring a room or the like. Here, it is assumed that the input point cloud data is corrected so that the horizontal is maintained with the vertical coordinate (Z coordinate) of the floor surface being 0 m. Also, it is assumed that the values of the point cloud data are acquired in known units such as meters.

[0050] Since the drawing data to be compared with the point cloud data is generally set as a cross-section assuming a certain height from the floor surface, the point cloud analysis unit 532 extracts the point cloud at a certain height from the floor surface from the point cloud data and uses it as the matching candidate point cloud. For example, when the floor plan corresponding to the drawing data is created as a cross-section at a height of 1 m from the floor surface, only the point cloud in the vicinity of a height of 1 m from the floor surface is extracted from the point cloud data. When the coordinates of a point belonging to the point cloud data expressed in the three axes of X, Y, and Z are (px, py, pz), it is preferable to extract only the points that satisfy the following two inequalities as the matching candidate point cloud in consideration of the accuracy of the point cloud data. Here, the height range h is preferably obtained experimentally according to the density and accuracy of the point cloud.

[0051] pz ≦ 1 + h / 2 pz > 1 - h / 2

[0052] FIG. 7 is a diagram showing an example of the matching candidate point cloud. By appropriately setting the height range h in the above inequality, the matching candidate point cloud is generated as point cloud data on the XY plane horizontal to the floor surface.

[0053] In this embodiment, the matching candidate images generated by the drawing analysis unit 531 and the matching of a plurality of matching candidate point clouds generated by the point cloud analysis unit 532 are performed to align the point cloud data. The plurality of matching candidate point clouds are, for example, point clouds acquired in a plurality of rooms in a building. Although there is no common part between the point clouds, by aligning each point cloud with the position of the drawing data, the alignment between the point clouds can be enabled.

[0054] Conventionally, a technique called registration has been used for the alignment of point cloud data. For example, there is a method in which corresponding points are obtained by an algorithm using the ICP (Iterative Closest Point) algorithm or the FPFH (Fast Point Feature Histograms) feature, and then a rigid body transformation for alignment is obtained by the RANSAC (Random Sample Consensus) algorithm. These are mainly methods for three-dimensional point clouds, but they can be used as degenerate cases in two dimensions, and in this embodiment, these methods can also be used to align the point cloud data.

[0055] However, since the ICP algorithm greatly depends on the initial value, there is a problem that it falls into a local minimum and the point cloud cannot be arranged at a desired position. Also, in the algorithm using the FPFH feature, since the point cloud data is acquired only inside the wall, while the drawing data has data on both the outside and inside of the wall, common features cannot be extracted, and there is a problem that the point cloud cannot be arranged at a desired position. In view of these, the alignment processing unit 530 in FIG. 4 uses a method of detecting a straight line portion from the drawing data and the point cloud data and arranging the point cloud at a desired position by alignment processing using the endpoints and intersection points of the straight line.

[0056] The pair selection unit 533 selects pairs of endpoints and intersection points of straight lines in each of the matching candidate images and the matching candidate point clouds. FIG. 8 is a diagram showing examples of endpoints and intersection points of straight lines. FIG. 8(a) is an example of a matching candidate image, and the white dots (white circles) in the figure indicate the endpoints and intersection points of straight lines. The pair selection unit 533 selects two of these white dots as candidate pairs for the matching candidate image. FIG. 8(b) is an example of a matching candidate point cloud, and the black dots (black circles) in the figure indicate the endpoints and intersection points of straight lines. The pair selection unit 533 selects two of these black dots as candidate pairs for the matching candidate point cloud.

[0057] The position estimation unit 534 matches the candidate pairs of the matching candidate image and the candidate pairs of the matching candidate point cloud, and estimates the position to relatively move the point cloud data to the corresponding part of the drawing data using the rigid body transformation unit 536 included in the position estimation unit 534. In the present embodiment, as will be described later, the selection of candidate pairs by the pair selection unit 533 and the estimation of the position by the position estimation unit 534 are repeatedly executed to determine the position where the point cloud is relatively moved and superimposed on the drawing data.

[0058] The display image generation unit 535 generates an image in which each point cloud (matching candidate point cloud) is superimposed on the drawing data according to the determined position of the point cloud. FIG. 9 is a diagram showing an example of a screen displayed on the display 308 of the terminal device 3. On the display screen 1000, a selection screen 1300 including a processing result area 1310, a translation button 1331, a rotation button 1332, and a confirmation button 1390 is displayed. Further, the processing result area 1310 includes a superimposition screen 1350 in which each point cloud is superimposed on the drawing data.

[0059] In the superimposition screen 1350 of FIG. 9, three point cloud data (first point cloud, second point cloud, third point cloud) are superimposed on the drawing data. The user can confirm the result of the alignment process by clicking the confirmation button 1390 using a pointer 1340 such as a mouse 312. Further, as will be described later, the result of the alignment process can be corrected for each point cloud by using the translation button 1331, the rotation button 1332, and the pointer 1340.

[0060] In FIG. 9, an example is shown in which a plurality of point cloud data are simultaneously superimposed on the drawing data. However, each point cloud may be superimposed separately. For example, first, only the first point cloud is superimposed on the drawing data and displayed. After the user clicks the confirmation button 1390, only the second point cloud is superimposed on the drawing data and displayed. Further, after the user clicks the confirmation button 1390, only the third point cloud may be superimposed on the drawing data and displayed.

[0061] FIG. 10 is a flowchart showing the procedure of the point cloud alignment process. First, the pair selection unit 533 selects a first candidate pair from the end points and intersection points of the matching candidate point cloud for one point cloud data, and selects a second candidate pair from the end points and intersection points of the matching candidate image (steps S100, S101).

[0062] Next, the position estimation unit 534 calculates the length L1 of the first candidate pair and the length L2 of the second candidate pair (step S102). Since the scale of the drawing data is known as described above and the matching candidate point cloud and the matching candidate image are created at the same scale, the lengths of the two pairs can be calculated and compared in the same unit (for example, meter).

[0063] If the difference in the lengths of the two pairs is large, the process returns to S101. If the difference is small, the process proceeds to S104 (step S103). When the flow returns to S101, the pair selection unit 533 selects the next second candidate pair (step S101). When comparing the lengths of the pairs, since the data contains errors, for example, when the following two inequalities are satisfied, it is determined that the difference in the lengths of the two pairs is small, and otherwise it is determined that the difference in the lengths of the two pairs is large. However, th is determined experimentally considering the data error.

[0064] L2 - th ≤ L1 L1 ≤ L2 + th

[0065] When the position estimation unit 534 determines that the difference in the lengths of two pairs is small, it obtains a rigid body transformation from the matching candidate point group to the matching candidate image such that the two pairs overlap (step S104). However, it is assumed that the rigid body transformation is only a parallel translation and rotation in the XY plane (horizontal plane).

[0066] Subsequently, the rigid body transformation unit 536 relatively moves the matching candidate point group with respect to the matching candidate image using the derived rigid body transformation. By this relative movement, all the points belonging to the matching candidate point group move. Then, the position estimation unit 534 totals the number of combinations in which the distance between a candidate pixel and a point is equal to or less than a certain value d for all the candidate pixels of the matching candidate image and all the points belonging to the matching candidate point group after the relative movement (step S105). Here, the certain value d is preferably obtained experimentally according to the resolution of the drawing data and the accuracy of the point group.

[0067] If the number of combinations totaled in S105 is greater than the maximum value of the number of combinations, the candidate transformation and the maximum value of the number of combinations are updated (steps S106, S107), and the flow proceeds to S108. In S107, the candidate transformation is updated to the rigid body transformation derived in S104, and the maximum value of the number of combinations is updated to the number of combinations totaled in S105. However, at the start of this flow, it is assumed that the candidate transformation is set to the identity transformation (a transformation that does not move all points), and the value of the maximum value of the number of combinations is set to zero. Note that the initial settings of the candidate transformation and the maximum value of the number of combinations may be other values. Also, in the above, in S105, the distance between the candidate pixel and the point is checked for all combinations of all candidate pixels and all points, but the distance may be checked for some combinations. For example, the number of combinations in which the distance between a candidate pixel and a point is equal to or less than a certain value d may be totaled for all the points belonging to the first candidate pair and all the candidate pixels belonging to the second candidate pair.

[0068] In S106, if the total number of combinations is less than or equal to the maximum number of combinations, the candidate transformation and the maximum number of combinations are not updated, and the flow proceeds to S108. The pair selection unit 533 checks whether all the second candidate pairs have been selected. If there are still unselected second candidate pairs, the flow returns to S101. If there are no unselected second candidate pairs, the flow proceeds to S109 (step S108).

[0069] Next, the pair selection unit 533 checks whether all the first candidate pairs have been selected. If there are still unselected first candidate pairs, the flow returns to S100. If there are no unselected first candidate pairs, the flow ends (step S109).

[0070] In this way, in the pair selection unit 533 and the position estimation unit 534, the procedures of S100 to S109 are repeated until there are no combinations of the first candidate pairs and the second candidate pairs. When the flow ends, the rigid body transformation set in the candidate transformation is determined as the relative movement used for alignment. In the alignment process, when using the above-described conventional algorithm, the pair selection unit 533 and the position estimation unit 534 may be replaced with an alignment unit according to the conventional method to constitute the alignment processing unit 530. In that case, the rigid body transformation of the matching candidate point group is obtained by the alignment unit according to the conventional method and is used for the relative movement in the display image generation unit 535 described later.

[0071] The image generation unit 535 generates an image in which the matching candidate point group relatively moved (transformed) by the rigid body transformation obtained by the pair selection unit 533 and the position estimation unit 534 is superimposed on the drawing data. The user can individually correct the result of the alignment process by clicking the parallel movement button 1331 or the rotation button 1332 and then dragging the position of the point group using the pointer 1340. For example, after the parallel movement button 1331 is clicked, if a certain point is dragged, the point group to which the point belongs may be parallelly moved. Also, after the rotation button 1332 is clicked, if a certain point is dragged, the point group to which the point belongs may be rotated around a predetermined point. The user can combine these operations while viewing the superimposed image to correct the result of the alignment process. When the user determines that the alignment result is correct based on the superimposed image, the user can click the confirmation button 1390 to confirm the final position of the point group data.

[0072] Note that the matching candidate point group is two-dimensional point group data on a horizontal plane. However, since the height alignment with respect to the drawing data is already completed, as described above, the alignment process is completed even as three-dimensional point group data by performing parallel movement in the horizontal plane and rotation around the vertical axis. Also, by performing the alignment process on each point group data corresponding to each room of the drawing data, the alignment process as three-dimensional point group data is completed for all rooms.

[0073] FIG. 11 is a sequence diagram showing an example of point group processing according to the present embodiment.

[0074] The reception unit 32 of the terminal device 3 receives an input operation related to user information (step S1). The transmission / reception unit 31 transmits a request for a setting screen including the user information received in step S1 to the management server 5 of the terminal device 3, and the transmission / reception unit 51 of the management server 5 receives the request transmitted from the terminal device 3. (step S2).

[0075] Next, the storage / reading unit 59 of the management server 5 searches the user information management DB5001 using the user information included in the request received in step S2 as a search key, reads out the file name of the three-dimensional point cloud data associated with the user information included in the request, and the generation unit 57 of the management server 5 generates a display screen including a setting screen based on the file name read by the storage / reading unit 59 (step S3).

[0076] This setting screen is a GUI (Graphical User Interface) screen on which images such as input means for point cloud information (file name of point cloud data, etc.), selection means for point cloud processing, and input means for drawing information (file name of drawing data, etc.) used for alignment processing are arranged.

[0077] The transmission / reception unit 51 transmits display screen information including the setting screen information related to the setting screen generated in step S3 to the terminal device 3, and the transmission / reception unit 31 of the terminal device 3 receives the display screen information transmitted from the management server 5 (step S4).

[0078] Next, the display control unit 34 of the terminal device 3 causes the display 308 to display the display screen including the setting screen received in step S4 (step S5). The reception unit 32 of the terminal device 3 receives a predetermined input operation of the user on the displayed setting screen. This input operation includes operations for inputting point cloud information, point cloud processing information, and drawing information. In the present embodiment, it is assumed that the user inputs information for instructing alignment processing and drawing information used for alignment processing as the point cloud processing information.

[0079] The transmission / reception unit 31 transmits input information related to the input operation received by the reception unit 32 to the management server 5, and the transmission / reception unit 51 of the management server 5 receives the input information transmitted from the terminal device 3 (step S6). This input information includes the input point cloud information, point cloud processing information, and drawing information.

[0080] The storage / reading unit 59 of the management server 5 searches the point cloud management DB5004 using the point cloud setting information included in the input information received in step S6 as a search key, and reads out the three-dimensional point cloud data and drawing data to be subjected to the alignment process.

[0081] Also, the storage / reading unit 59 searches the setting information management DB5002 using the point cloud processing information included in the input information received in step S6 as a search key, and reads out the point cloud processing program (in this embodiment, the alignment processing program).

[0082] The processing unit 53 of the management server 5 generates point cloud processing information (step S7) based on the three-dimensional point cloud data, drawing data, and alignment processing program read from the storage / reading unit 59. The point cloud processing information is information on the result of subjecting the point cloud data to the alignment process.

[0083] Step S7 is an example of a point cloud processing step for executing an alignment process on point cloud data indicating a three-dimensional point cloud.

[0084] The generation unit 57 of the management server 5 generates a display screen including the display of the point cloud processing information (result of the alignment process) and a selection screen that allows selection of whether to end the alignment process or execute it again. The selection screen is a GUI (Graphical User Interface) screen such as the selection screen 1300 shown in FIG. 9. Also, the transmission / reception unit 51 transmits the generated display screen information to the terminal device 3 (step S8). Step S8 is an example of a generation step.

[0085] The transmission / reception unit 31 of the terminal device 3 receives the display screen information transmitted from the management server 5, the display control unit 34 of the terminal device 3 causes the received display screen to be displayed on the display 308, and the reception unit 32 of the terminal device 3 receives a predetermined input operation of the user with respect to the displayed display screen (step S9).

[0086] This input operation includes a selection operation for selecting the end or re-execution of the alignment process, and an adjustment operation for adjusting the processing result of the alignment process. The selection operation may be one that selects one result from a plurality of alignment results.

[0087] The transmission / reception unit 31 transmits input information related to the input operation received by the reception unit 32 to the management server 5, and the transmission / reception unit 51 of the management server 5 receives the input information transmitted from the terminal device 3 (step S10).

[0088] When this input information includes selection information indicating the end of the alignment process, the processing unit 53 of the management server 5 determines the processing result of the alignment process.

[0089] On the other hand, when the input information includes selection information indicating the re-execution of the alignment process or adjustment information by the adjustment operation, the processing unit 53 of the management server 5 re-executes the alignment process in step S7 based on this information.

[0090] The processing unit 53 converts the processing result information into a file format readable by point cloud processing software, a file format readable by 3D-CAD software, a file format readable by BIM / CIM software, etc., and the storage / reading unit 59 stores the converted processing result information in the processing result management DB5005, the recording medium 506, or the CD-RW513 (step S11). Step S11 is an example of a storage step.

[0091] The transmission / reception unit 51 transmits the determined processing result information to the terminal device 3 (step S12).

[0092] The transmission / reception unit 31 of the terminal device 3 receives the processing result information transmitted from the management server 5, and the display control unit 34 of the terminal device 3 causes the received processing result to be displayed on the display 308. (Step S13).

[0093] As described above, the functions of the management server 5 in FIG. 3 may be integrated into the terminal device 3, and the terminal device 3 may also execute the processing of the management server 5 in FIG. 11.

[0094] FIG. 12 is an explanatory diagram showing an example of a setting screen according to the present embodiment. FIG. 12 shows a display screen 1000 displayed on the display 308 of the terminal device 3 in step S5 of the sequence diagram shown in FIG. 11.

[0095] The display control unit 34 of the terminal device 3 causes the user information display screen 1100 and the setting screen 1200 to be displayed on the display screen 1000.

[0096] The setting screen 1200 includes a point cloud setting screen 1210, a processing setting screen 1220, and an execution button 1230.

[0097] The point cloud setting screen 1210 is a screen for receiving a point cloud setting operation for setting point cloud data indicating a three-dimensional point cloud used for executing point cloud processing. The display control unit 34 causes the point cloud setting areas 1212 and 1214 to be displayed in association with the file names of the plurality of point cloud data read by the storage / reading unit 59. The point cloud setting areas 1212 and 1214 can be set in plurality.

[0098] The processing setting screen 1220 is a screen for receiving a setting operation for setting the type of point cloud processing. The display control unit 34 causes the processing setting areas 1221, 1222, and 1223 to be displayed in association with the names of the plurality of point cloud processes. When setting the alignment process, the drawing setting areas 1225 and 1226 are displayed in association with the file names of the respective drawing data. The drawing setting areas 1225 and 1226 can be set in plurality. Further, the display control unit 34 causes the execution button 1230 for determining various setting operations to be displayed.

[0099] Furthermore, the display control unit 34 displays a pointer 1240 such as a mouse 312 for selecting the processing setting areas 1221 to 1223 and the drawing setting areas 1225 and 1226.

[0100] When various setting areas are clicked by the pointer 1240, the reception unit 32 of the terminal device 3 causes the display control unit 34 to display black circles and dots in the various setting areas as shown in the figure. The reception unit 32 accepts various setting operations. When the execution button 1230 is operated, the various setting operations are completed and the alignment process is executed.

[0101] Specifically, as described in steps S6 and S7 of FIG. 11, the transmission / reception unit 31 transmits input information including various setting information based on the various setting operations received by the reception unit 32 to the management server 5, and the processing unit 53 executes the alignment process.

[0102] As described above, FIG. 9 is an explanatory diagram showing an example of a selection screen according to the present embodiment. FIG. 9 shows a display screen 1000 displayed on the display 308 of the terminal device 3 in step S9 of the sequence diagram shown in FIG. 11, and the display control unit 34 of the terminal device 3 causes the selection screen 1300 to be displayed on the display screen 1000.

[0103] In this way, by using the setting screen 1200 and the selection screen 1300, the user can execute the alignment process on the desired point cloud data and can easily correct the result of the alignment using the mouse 312 or the like.

[0104] In the present embodiment, an example has been described in which, for each point cloud data, only the alignment result of one point cloud data is superimposed and displayed on the drawing data. However, it is also possible to present a plurality of alignment candidates for one point cloud data to the user. For example, a plurality of rigid body transformations corresponding to those with a large number of combinations tabulated in S105 of FIG. 10 are left as candidates, and the point cloud data relatively moved by each rigid body transformation can be simultaneously or alternately displayed on the superimposed screen 1350. The user can determine the position of the point cloud data by selecting while checking each position on the superimposed screen 1350.

[0105] As described above, according to the present embodiment, by aligning a plurality of point clouds with the drawing data, even when there is no common part between the point clouds, the alignment of the plurality of point clouds can be appropriately performed. Further, after executing the point cloud alignment process, if the alignment does not reach the desired quality, the burden on the user for post-processing can be reduced by adjusting the position of the point cloud while checking the point cloud superimposed on the drawing data.

[0106] (Second Embodiment) In the first embodiment, when there are a plurality of rooms with the same shape in the drawing data, it is difficult to determine to which room to align the matching candidate point clouds having a shape similar to the shape of those rooms. In the second embodiment, if the positions of the doors and windows of such a plurality of rooms are not the same, the matching candidate point clouds can be aligned.

[0107] FIG. 13 is a functional block diagram of the alignment processing unit 530 according to the second embodiment. The alignment processing unit 530 in the present embodiment includes a drawing analysis unit 531, a point cloud analysis unit 532, an attribute addition unit 537, a pair selection unit 533, a position estimation unit 534, and a display image generation unit 535. Hereinafter, differences from the first embodiment will be described.

[0108] The operations of the drawing analysis unit 531 and the point cloud analysis unit 532 are the same as those in the first embodiment. The attribute addition unit 537 adds attribute information such as walls, doors, and windows to the matching candidate image generated by the drawing analysis unit 531 and the matching candidate point cloud generated by the point cloud analysis unit 532. In the case of drawing data, the attribute information is determined by which part of the drawing the candidate pixel of the matching candidate image is extracted from. FIGS. 14(a) and (b) are diagrams showing an example of the attribute information added to each candidate pixel of the matching candidate image. Here, the circled marks indicate the attribute information added to the pixels classified as walls, the cross marks indicate the attribute information added to the pixels classified as doors, and the square marks indicate the attribute information added to the pixels classified as windows. Category numbers such as 1, 2, 3, etc. are associated with these attribute information.

[0109] In the case of point cloud data, the attribute information can be classified using a technique called point cloud segmentation technology. FIG. 14(c) is a diagram showing an example of the attribute information added to each point of the matching candidate point cloud. Here, the circle marks indicate the points classified as walls, the cross marks indicate the points classified as doors, and the square marks indicate the attribute information added to the points classified as window frames. To these attribute information, category numbers such as 1, 2, 3, etc., similar to the attribute information added to the matching candidate image, are associated. The attribute information in the point cloud data is an example of a point cloud marker for the user to align the point cloud data with the drawing data.

[0110] The operations of the pair selection unit 533 and the position estimation unit 534 are the same as those in the first embodiment. However, in S105 of FIG. 10, the condition for aggregating the number of combinations is added with the attribute information of the points after the relative movement by rigid body transformation and the attribute information of the candidate pixels being the same category number. As a result, even if the shape of the matching candidate point cloud is the same as the shapes of the plurality of rooms in the drawing data, for the rooms where the attribute information of the candidate pixels and the attribute information of the points corresponding to the candidate pixels are different, the number of combinations to be aggregated becomes small, and the alignment to that room is not performed. Also, among such a plurality of rooms, alignment to the room where the attribute information of the candidate pixels and the points most match (the aggregated number of combinations becomes the maximum value) is performed, so that appropriate alignment processing becomes possible.

[0111] The operation of the display image generation unit 535 is the same as that in the first embodiment. As shown in FIG. 9, a superimposed screen 1350 is generated by aligning and superimposing a plurality of point cloud data on each room of the drawing data, and is displayed on the display screen 1000.

[0112] In the display image generation unit 535, the display control unit 34 of the terminal device 3 may cause attribute information to be displayed at points superimposed on the drawing data in the superimposed screen 1350. For example, icon images such as circles at points classified as walls, crosses at points classified as doors, and squares at points classified as window frames can be superimposed to indicate the attribute information added to each. FIG. 15 is a diagram showing a part of the superimposed screen 1350 on which the attribute information is displayed. The user can check the displayed attribute information to determine whether the point group matches the positions of doors and windows on the drawing. If it is determined that they do not match, the result of the alignment process can be corrected using the parallel movement button 1331 and the rotation button 1332.

[0113] As described above, according to the present embodiment, by adding attribute information to the drawing data and the point cloud data and performing alignment with reference to the attribute information in addition to the respective shapes, even when there are multiple rooms with similar shapes, it is possible to accurately align multiple point clouds to the drawing data. When the user aligns the point cloud data with the drawing data, since the attributes of the drawing data may be visually confirmed in some cases, as a modification of the present embodiment, attribute information may be added to the point cloud data, but it may not be necessary to add attribute information to the drawing data.

[0114] (Third Embodiment) In the above-described embodiment, the scale of the drawing data was known, but in reality, there may be cases where the scale is unknown. In this embodiment, even when the scale of the drawing data is unknown, by setting a provisional scale, it is possible to accurately perform the alignment process of the point cloud.

[0115] FIG. 16 is a functional block diagram of the alignment processing unit 530 according to the third embodiment, and FIG. 17 is a flowchart showing the procedure of the alignment process of the point cloud. The alignment processing unit 530 in the present embodiment is different from the first embodiment in that the position estimation unit 534 further includes a correction value derivation unit 538. Hereinafter, the configuration and processing procedure of the present embodiment will be described with reference to FIGS. 16 and 17. Detailed description of the same parts as those in the first embodiment will be omitted.

[0116] The operations of the drawing analysis unit 531, the point group analysis unit 532, and the pair selection unit 533 are the same as those in the first embodiment. Also in this embodiment, the pair selection unit 533 repeatedly executes the selection of candidate pairs and the position estimation by the position estimation unit 534 to estimate the position where the point group is superimposed on the drawing data.

[0117] First, after the pair selection unit 533 selects the first candidate pair and the second candidate pair (steps S170, S171), the position estimation unit 534 calculates the length of the second candidate pair based on a provisional scale (step S172). For example, when the length of one pixel of the matching candidate image based on the provisional scale is = 0.01 m, the length of 120 pixels is calculated as 120 × 0.01 = 1.2 m. Here, if the provisional scale is too large or too small compared to the correct value, it becomes difficult to compare the lengths at the position estimation unit 534 because the criteria for the distance obtained from the pair of candidate pixels and the distance obtained from the pair of points in the matching candidate point group are significantly different. Therefore, it is desirable to use an appropriate value for the provisional scale. For example, since the width of a room door is approximately determined as a standard, the provisional scale can be set using the width of the door on the drawing data as a clue.

[0118] Next, the position estimation unit 534 compares the difference between the length L1 of the first candidate pair and the length L2 of the second candidate pair (step S173). Considering the accuracy of the provisional scale, it is preferable that the threshold value used for comparing the lengths of the pairs is a value larger than that used in the first embodiment. For example, when the following two inequalities are satisfied, it is determined that the difference in the lengths of the two pairs is small, and otherwise, it is determined that the difference in the lengths of the two pairs is large. Here, α (where α > 1) is a coefficient for making the threshold value used for comparing the lengths of the pairs larger than that used in the first embodiment, and it is determined experimentally considering the accuracy of the provisional scale.

[0119] L2 - α × th ≤ L1 L1 ≤ L2 + α × th

[0120] When it is determined that the difference in the lengths of the two pairs is small, the position estimation unit 534 assumes that the length L1 of the first candidate pair is equal to the length L2 of the second candidate pair, and obtains a correction value k for correcting the temporary scale using the following formula (step S174).

[0121] k = L1 / L2

[0122] The corrected temporary scale is obtained by multiplying the temporary scale by k. For example, if the length of one pixel of the matching candidate image based on the temporary scale is = 0.01 m and the correction value k is 1.3, the length of 120 pixels is calculated as 120 × 0.01 × 1.3 = 1.56 m.

[0123] Based on the corrected temporary scale, the position estimation unit 534 obtains a rigid body transformation from the matching candidate point group to the matching candidate image such that the two pairs overlap (step S175).

[0124] When the position estimation unit 534 totals the number of combinations in which the distance between the candidate pixel and the point is equal to or less than a certain value d, it also calculates the distance based on the corrected temporary scale (step S176). If the total number of combinations is greater than the maximum value of the number of combinations, the candidate correction value, the candidate transformation, and the maximum value of the number of combinations are updated (steps S177, S178), and the flow proceeds to S179. Here, the candidate correction value is updated to the correction value derived in S174, the candidate transformation is updated to the rigid body transformation derived in S175, and the maximum value of the number of combinations is updated to the number of combinations tabulated in S176. It is assumed that the candidate correction value is set to 1 at the start of this flow.

[0125] The pair selection unit 533 and the position estimation unit 534 execute the above processing for all combinations of the first candidate pair and the second candidate pair, and determine the candidate correction value at the end of this flow as the correction value for correcting the temporary scale. In this way, a correction value for correcting the temporary scale is obtained for one point group.

[0126] The alignment processing unit 530 obtains a correction value ki for each of a plurality of point clouds i for which alignment is performed by the above-described processing, and obtains a final correction value kF using these. Here, i (i = 1, 2, 3,...) is the number of the point cloud. The final correction value kF can be obtained as the average of the correction values obtained for each point cloud. Note that, in order to exclude the influence of correction values with large errors, the final correction value kF may be obtained by taking the median value.

[0127] Next, the alignment processing unit 530 sets the final scale to kF times the provisional scale, and again executes the alignment processing for each point cloud using the final scale.

[0128] The operation of the display image generation unit 535 is the same as that in the first embodiment. As shown in FIG. 9, a superimposed screen 1350 is generated by aligning and superimposing a plurality of point cloud data on each room of the drawing data, and is displayed on the display screen 1000. The display image generation unit 535 displays the drawing data with its size changed (enlarged or reduced) relative to the point cloud data using the final scale. Alternatively, instead of changing the size of the drawing data, the display image generation unit 535 may change the size (enlarge or reduce) of the point cloud data relative to the drawing data using the reciprocal of the final scale. Note that, in the present embodiment, since the scale of the drawing data is not known, buttons for performing "enlarge / reduce" may be arranged in the selection screen 1300 so that the user can correct the scale using these buttons while checking the superimposed screen 1350.

[0129] Note that, in the above description, the third embodiment is realized by adding the correction unit derivation unit 358 to the first embodiment, but it may also be realized by adding the correction unit derivation unit 358 to the second embodiment. In that case, the attribute addition unit 537 adds attribute information corresponding to the category number to the matching candidate image and the matching candidate point cloud, and in S176 of FIG. 17, it is only necessary to add that the attribute information of the points after relative movement by rigid body transformation and the attribute information of the candidate pixels are the same category number to the condition for totaling the number of combinations.

[0130] Thus, according to this embodiment, even when the scale of the drawing data is unknown, a temporary scale is set and the temporary scale is corrected during the alignment process, so that the alignment process of the point cloud can be accurately performed.

[0131] Aspects of the present invention are as follows, for example. <1> A point cloud processing apparatus including display screen generation means for generating a display screen in which a plurality of point clouds indicating a three-dimensional point cloud obtained by measuring an object are superimposed on an image indicating the object. <2> The point cloud processing apparatus according to <1>, wherein the display screen generation means generates the display screen using relative movement for relatively moving at least one of the plurality of point clouds with respect to the image. <3> The point cloud processing apparatus according to <1> or <2>, wherein the display screen generation means generates the display screen using size change for relatively changing the size of at least one of the plurality of point clouds with respect to the image. <4> The point cloud processing apparatus according to any one of <1> to <3>, wherein the display screen generation means generates the display screen in which a point cloud marker for aligning with the image is displayed for at least one of the plurality of point clouds. <5> The point cloud processing apparatus according to any one of <1> to <3>, wherein the display screen generation means generates the display screen in which attribute information is displayed for at least one of the plurality of point clouds. <6> The point cloud processing apparatus according to any one of <1> to <5>, including display control means for displaying the display screen. <7> The point cloud processing apparatus according to any one of <2> to <6>, wherein the relative movement includes at least one of parallel movement in a horizontal plane and rotation in a horizontal plane. <8> The point cloud processing apparatus according to any one of <3> to <7>, wherein the size change includes at least one of enlargement and reduction. <9> The point cloud processing apparatus according to any one of <5> to <8>, wherein the attribute information is information indicating at least one of a door and a window frame. <10> This is a point cloud processing method that executes a display screen generation step of generating a display screen in which a plurality of point clouds representing a three-dimensional point cloud obtained by measuring an object are superimposed on an image showing the object. <11> This is a program that causes a computer to execute a display screen in which a plurality of point clouds representing a three-dimensional point cloud obtained by measuring an object are superimposed on an image showing the object. <12> A point cloud processing system comprising a point cloud processing device and a terminal device capable of communicating with the point cloud processing device, wherein the point cloud processing device comprises a display screen generation means for generating a display screen in which a plurality of point clouds representing a three-dimensional point cloud obtained by measuring an object are superimposed on an image representing the object, and a transmission means for transmitting display screen information representing the display screen to the terminal device, and the terminal device comprises a receiving means for receiving the display screen information transmitted from the point cloud processing device and a display control means for displaying the display screen on a display unit. [Explanation of symbols]

[0132] 1. Point cloud processing system 3 Terminal Devices 5 Management Server 32 Reception Department 34 Display control unit 39 Memory / readout section 53 Processing section 57 Generation part 100 Communication Network 308 Display 311 Keyboard 312 Mouse 530 Alignment processing unit 531 Drawing Analysis Department 532 Point cloud analysis section 533 Pair Selection Unit 534 Position estimation part 535 Display image generation unit 536 Rigid body transformation part 537 Attribute Addition Section 538 Correction value derivation part 1300 Selection Screen 1310 Processing Result Area 1331 Parallel movement button 1332 Rotation button 1350 Overlay screen

Prior art documents

Patent documents

[0133]

Patent Document 1

Claims

1. A point cloud processing apparatus comprising display screen generation means for generating a display screen in which a plurality of point clouds indicating a three-dimensional point cloud obtained by measuring an object are superimposed on an image indicating the object.

2. The point cloud processing apparatus according to claim 1, wherein the display screen generation means generates the display screen using relative movement for relatively moving at least one of the plurality of point clouds with respect to the image.

3. The point cloud processing apparatus according to claim 1, wherein the display screen generation means generates the display screen using size change for relatively changing the size of at least one of the plurality of point clouds with respect to the image.

4. The point cloud processing apparatus according to any one of claims 1 to 3, wherein the display screen generation means generates the display screen in which a point cloud marker for aligning with the image is displayed for at least one of the plurality of point clouds.

5. The point cloud processing apparatus according to any one of claims 1 to 3, wherein the display screen generation means generates the display screen in which attribute information is displayed for at least one of the plurality of point clouds.

6. The point cloud processing apparatus according to any one of claims 1 to 3, further comprising display control means for displaying the display screen.

7. The point cloud processing apparatus according to claim 2, wherein the relative movement includes at least one of parallel movement in a horizontal plane and rotation in a horizontal plane.

8. The point cloud processing apparatus according to claim 3, wherein the size change includes at least one of enlargement and reduction.

9. The point cloud processing apparatus according to claim 5, wherein the attribute information is information indicating at least one of a door and a window frame.

10. A point cloud processing method for executing a display screen generation step of generating a display screen in which a plurality of point clouds indicating a three-dimensional point cloud obtained by measuring an object are superimposed on an image indicating the object.

11. A program for causing a computer to execute generating a display screen in which a plurality of point clouds indicating a three-dimensional point cloud obtained by measuring an object are superimposed on an image indicating the object.

12. A point cloud processing system comprising a point cloud processing apparatus and a terminal device capable of communicating with the point cloud processing apparatus, wherein the point cloud processing apparatus comprises display screen generation means for generating a display screen in which a plurality of point clouds indicating a three-dimensional point cloud obtained by measuring an object are superimposed on an image indicating the object, and transmission means for transmitting display screen information indicating the display screen to the terminal device, and the terminal device Receiving means for receiving the display screen information transmitted from the point cloud processing device; A point cloud processing system comprising display control means for displaying the display screen on a display unit.

Images