Piping display system and piping display method

Abstract

To provide a piping display system and a piping display method that superimpose CAD data onto images of a piping installation site.SOLUTION: A piping display system comprises: a display part that displays captured images and CAD piping images on a screen; an input part that receives designated alignment points from a reference point group and a CAD point group displayed on the screen; a reference point group alignment part that performs alignment between the reference point group and the CAD point group using the alignment points and determines a first coordinate transformation matrix representing the coordinate transformation matrix between the CAD point group and the reference point group; a measurement point group alignment part that extracts a measurement point group from a piping image, performs alignment between the measurement point group and the reference point group, determines a second coordinate transformation matrix representing the coordinate transformation matrix between the measurement point group and the reference point group, and further determines a third coordinate transformation matrix representing the coordinate transformation matrix between the CAD point group and the measurement point group by multiplying the first and second coordinate transformation matrices; and a display processing part that superimposes CAD data onto the piping image using the third coordinate transformation matrix.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a piping display system and a piping display method. [Background technology]

[0002] In process piping line checks, it is necessary to confirm that the position and orientation of pipes, valves, etc. were installed according to the drawings when the plant was constructed, and that process requirements are being observed.

[0003] During piping line checks, a person inspects and records the installed piping based on the P&ID (Piping & Instrumentation Diagram), construction drawings, and CAD drawings.

[0004] A P&ID is a diagram showing the piping process and its connections to other equipment and instruments using standard dedicated P&ID symbols.

[0005] Therefore, piping line checks require specialized knowledge of P&IDs, and also require matching the drawings with the installed positions of the piping, etc. This requires highly skilled experts and requires a huge amount of man-hours.

[0006] Furthermore, when checking the process piping line, it is important to check whether the installation positions and model numbers of valves and meters have been installed according to the drawings and to keep a record of this.

[0007] There is a need for ways to make such work as efficient as possible and reduce the number of steps required.

[0008] Patent Document 1 discloses a drawing projection system that can project drawing information in real space while adjusting the position and scale.

[0009] The drawing projection system displays three-dimensional drawing data superimposed on real space.

[0010] The drawing projection system 100 includes a space scanning means for scanning the real space and recognizing the spatial shape, a projection image generating means for scaling the drawing data to fit the real space, and a projection means for mapping and displaying the scaled drawing data onto the real space.

[0011] Patent Document 2 discloses a building management system.

[0012] This building management system includes a position determination device that determines the location where a photograph of a building was taken from the photograph, and a learning device that uses machine learning to generate the criteria used by the position determination device to determine the location.

[0013] The learning device includes a three-dimensional shape data storage unit that stores three-dimensional shape data that models a building, a two-dimensional image generation unit that generates a two-dimensional image of the building by setting a viewpoint position and a line of sight and rendering the three-dimensional shape data, and a learning unit that generates a judgment criterion by learning the relationship between extracted information extracted from the two-dimensional image generated by the two-dimensional image generation unit and the viewpoint position that was set when generating the two-dimensional image.

[0014] The position determination device includes a photo receiving unit that receives a photo of a building, and a determination unit that determines the position where the photo was taken based on a determination criterion generated by the learning device. [Prior art documents] [Patent documents]

[0015] [Patent Document 1] Japanese Patent Application Publication No. 2018-163466 [Patent Document 2] JP 2019-175144 A Non-patent document

[0016] [Non-Patent Document 1] LEPETIT, Vincent, et al. Fully automated and stable registration for augmented reality applications. In: The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings. IEEE, 2003. p. 93-102. [Non-patent document 2] BESL, Paul J.; MCKAY, Neil D. Method for registration of 3-D shapes. In: Sensor fusion IV: control paradigms and data structures. SPIE, 1992. p. 586-606. [Non-patent document 3]

[16] RUSU, RB; BLODOW, N.; BEETZ, M. Fast point feature histograms (FPFH) for 3D registration, ICRA'09 on. IEEE Press, 2009, 7: 18481853. Summary of the Invention [Problem to be solved by the invention]

[0017] The invention disclosed in Patent Document 1 is an invention that displays drawing data by superimposing it onto real space, adjusting the position and scale. This invention explains that it projects structural members that make up a building, such as pillars and roofs, but this is insufficient for checking whether the plant's process requirements are being met.

[0018] For example, there is no disclosure about checking the installation position and orientation of the installed valves, or the model numbers of the packings and flanges.

[0019] The invention disclosed in Patent Document 2 is an invention for appropriately managing buildings. It determines the position of the photographer from a photograph by learning two-dimensional images obtained from three-dimensional shape data.

[0020] However, piping inspections are carried out at plants under construction, and the photographs taken often differ from the blueprints, making it impossible to train the system.

[0021] A method of using augmented reality to check piping lines is being considered. In other words, piping line checks can be performed by overlaying 3D data on on-site video in real time. Using augmented reality makes it possible to intuitively check piping lines.

[0022] Therefore, in order to apply augmented reality to piping line inspection, it is necessary to align the image of the installed piping with the CAD data.

[0023] An edge-based method was used to align CAD data and on-site images by matching the edges of the CAD data with the edges extracted from the on-site images (Non-Patent Document 1).

[0024] However, there is a problem that alignment is difficult due to the large number of edges that occur in piping line inspection. For this reason, a point cloud-based method is considered as an alignment method instead of an edge-based method.

[0025] Point cloud-based methods use RGBD cameras that can measure depth to reconstruct the 3D shape of the site where the piping is installed.

[0026] By aligning the reconstructed point cloud with the point cloud generated from CAD data (hereafter referred to as CAD point cloud), it becomes possible to display them superimposed.

[0027] There are two types of point cloud-based registration methods: local and global. As shown in Non-Patent Document 2, the local method is a method using the Iterative Closest Point (ICP) algorithm.

[0028] Although this method requires setting an initial position, it is robust and can be applied to a variety of point cloud data, including the measured shape of the pipe installation site and CAD point clouds.

[0029] On the other hand, global registration methods have also been proposed. Global registration methods such as those shown in Non-Patent Document 3 do not require setting initial values ​​by using the characteristics of 3D shapes. For this reason, they are suitable for on-site superimposed display and augmented reality.

[0030] However, there is a problem in that the global method is difficult to apply to CAD point clouds and point clouds measured at the site where the piping has been installed. This is thought to be because the properties of the measured point clouds at the site and the CAD point clouds are different, making it difficult to find correspondences between feature points.

[0031] An object of the present invention is to efficiently align CAD data with pipes at a site where the piping has been installed, and to superimpose the CAD data on a photographed image of the site. [Means for solving the problem]

[0032] The above object can be achieved by a piping display system that superimposes and displays a CAD piping image created from CAD data of the piping on a photographed image, the piping display system comprising: a display unit that displays the photographed image and the CAD piping image on a screen; an input unit that accepts the specification of alignment points from the reference point cloud and CAD point cloud displayed on the screen; a reference point cloud alignment unit that aligns the reference point cloud and CAD point cloud using the accepted alignment points and obtains a first coordinate transformation matrix that is a coordinate transformation matrix between the CAD point cloud and the reference point cloud; a measurement point cloud alignment unit that obtains a measurement point cloud from the photographed image, aligns the measurement point cloud with the reference point cloud, obtains a second coordinate transformation matrix that is a coordinate transformation matrix between the measurement point cloud and the reference point cloud, and multiplies the first coordinate transformation matrix by the second coordinate transformation matrix to obtain a third coordinate transformation matrix that is a coordinate transformation matrix between the CAD point cloud and the measurement point cloud; and a display processing unit that superimposes and displays the CAD data on the photographed image using the third coordinate transformation matrix. [Effects of the Invention]

[0033] According to the present invention, CAD data can be superimposed on a photographed image of a piping installation site. [Brief explanation of the drawings]

[0034] [Figure 1] 1 is a block diagram illustrating an example of a system configuration of a piping display system according to an embodiment of the present invention. [Figure 2] 1 is a diagram illustrating an example of hardware for a tablet-type piping line check system according to an embodiment of the present invention. [Figure 3] FIG. 1 is a diagram illustrating an example of piping. [Figure 4] 10 is a display example of a CAD point cloud in an embodiment of the present invention. [Figure 5] 10 is a diagram illustrating an example of a display of a reference point group according to an embodiment of the present invention. [Figure 6] 10 is a display example of a measurement point cloud in an embodiment of the present invention. [Figure 7] 10 is a display example of a result of alignment between CAD data and a measurement point cloud in an embodiment of the present invention. [Figure 8] 10 is an example of a result of superimposed display in the embodiment of the present invention. [Figure 9] 10 is a flowchart illustrating an example of processing by a reference point group alignment unit according to an embodiment of the present invention. [Figure 10] 10 is a flowchart illustrating an example of processing by a measurement point cloud alignment unit in the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0035] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0036] In each drawing for explaining the embodiments, the same components are given the same names and symbols as much as possible, and repeated explanations thereof will be omitted.

[0037] The present invention is not limited to the following examples, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above examples have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations.

[0038] Furthermore, the processing units and processing modules described in the embodiments may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by having a processor interpret and execute a program that realizes each function.

[0039] The information explained in the embodiment may be a table, a database (DB), or data stored in the main memory.

[0040] FIG. 1 is a block diagram showing an example of the system configuration of a piping display system according to an embodiment of the present invention.

[0041] The piping display system 1 can be configured as a single computer system. The piping display system 1 includes a CPU (Central Processing Unit) 2, an input unit 3, a display unit 4, a camera 5, a memory 6, and an auxiliary storage device 7, which are connected via a bus 8.

[0042] The CPU 2 controls the operation of the piping display system 1. The CPU 2 loads a predetermined computer program stored in the auxiliary storage device 7 into the memory 6 and executes it, thereby realizing each function of the piping display device.

[0043] The input unit 3 is a device for a user to input instructions and information to the piping display system 1. The input unit 3 is configured, for example, as a manual switch, an operation button, an operation controller, a touch panel, a voice recognition device, or the like.

[0044] The display unit 4 is a device that provides information from the piping display system 1 to the user, and is a display device or a liquid crystal panel.

[0045] The camera 5 is a color digital camera such as a CCD (Charge Coupled Device) camera or a CMOS (Complementary MOS) camera, and outputs the captured image as digital data. The camera 5 can be equipped with a function to measure the distance to the object based on the time it takes for a projected laser beam to return from the object, or on the principle of pattern irradiation or stereo camera. Such a camera is referred to here as an RGBD camera.

[0046] The memory 6 is a storage device including, for example, a ROM (Read Only Memory) and a RAM (Random Access Memory).

[0047] The memory 6 stores software modules for a reference point cloud alignment unit 11 that aligns the CAD data 21 with the reference point cloud information 14, a measurement point cloud alignment unit 12 that aligns the CAD data 21 with the measurement point cloud information 16, and a display processing unit 19 that displays an image in which the CAD data is superimposed on a photographed image of the piping.

[0048] The auxiliary storage device 7 is a non-volatile storage device with a relatively large capacity, such as a hard disk or a flash memory device, etc. The auxiliary storage device 7 may also include a removable recording medium.

[0049] CAD data 21 of the piping (CAD data) is stored in the auxiliary storage device 7. Also, piping check information 22 for checking the piping is saved.

[0050] These pieces of information may be stored on a cloud that can be accessed via a network, which allows the CAD data 21 and piping check information 22 to be updated in a short period of time.

[0051] These software modules are executed by the CPU 2. These software modules may be realized as hardware.

[0052] When the data is stored in an auxiliary storage device, there is an advantage that work can be carried out without being affected by communication quality even in a building with a poor communication environment.

[0053] The auxiliary storage device 7 may store a computer program. For example, the reference point cloud alignment unit 11 and the measurement point cloud alignment unit 12 may be stored in the auxiliary storage device 7.

[0054] Reference point clouds are intermediate data used to match CAD data with captured images. There are two methods for creating reference point clouds: from CAD data or from captured images.

[0055] When creating a reference point group from CAD data, the polygons that make up each face of CAD data such as a surface model are divided into meshes using techniques such as FEM (finite element method), and points that represent the meshes are found.

[0056] Next, the ICP algorithm is used to match the measured point cloud, which is a point cloud captured by a depth camera, etc. In this case, the reference points created from the CAD data must be large enough to allow for errors.

[0057] When creating a reference point cloud from a captured image, the measurement information of the pipe from the current camera position obtained from the measurement points in the captured image using a method such as SLAM (Simultaneous Localization and Mapping) is used as the reference point cloud.

[0058] Next, the reference point cloud and CAD data are matched using FPFH.

[0059] If the reference point cloud is stored in the auxiliary storage device 7 as reference point cloud information 14, it becomes possible to superimpose the CAD data onto the captured image by matching the reference point cloud with CAD data (CAD point cloud) and by matching the reference point cloud with the measurement point cloud obtained from the captured image taken by the camera.

[0060] In this case, if attribute information indicating whether the reference point group was created from CAD data or from a photographed image is added to the reference point group information 14, matching can be performed based on the attribute information, making it possible to superimpose CAD data onto the photographed image with greater accuracy.

[0061] FIG. 2 shows an example of hardware of a tablet-type piping line check system according to an embodiment of the present invention.

[0062] It is a so-called tablet computer, and has a camera 5 on the back of the main body 31 and a touch panel 32.

[0063] 3 to 8, examples of flowcharts showing the processing in the embodiment of the present invention shown in FIGS. 9 and 10 will be described.

[0064] Figure 3 shows an example of piping. As shown in this figure, various pipes, instruments, devices, etc. are arranged at the construction site.

[0065] 3, a piping line 301 to be checked is superimposed on the photographed image, and the positions of a valve 302 and a meter 302 to be checked are also superimposed.

[0066] FIG. 9 is an example of a flowchart showing the processing of the reference point group alignment unit in the embodiment of the present invention.

[0067] First, CAD point cloud information is generated from the CAD data and stored in the CAD point cloud information 13 (S3). The CAD data is composed of a surface model, and a point cloud is generated from the surface data.

[0068] Specifically, the number of points to be generated per unit area of ​​the surface model is specified to generate the point cloud.

[0069] An example of a CAD point cloud display is shown in Figure 4. Next, the area around the piping is photographed using camera 5 (S4). The camera may be an RGB camera or an RGBD camera. From the photographed image, a 3D reconstruction method is used to perform 3D reconstruction of the area around the piping, and a reference point cloud is created and stored in reference point cloud information 14 (S5).

[0070] The reconstructed point cloud is called the reference point cloud. An example of the reference point cloud display is shown in Figure 5. Next, the corresponding points are accepted from the user to align the reference point cloud with the CAD point cloud (S6). In other words, the inspector interactively selects three alignment points, points 1 to 3, for the CAD point cloud (Figure 4).

[0071] At this time, the system determines what kind of equipment the specified alignment point is, such as a pipe, device, or measuring instrument, and assigns a weight to the alignment point according to the equipment. When determining the priority of using the specified alignment point for alignment, selecting the alignment point based on the assigned weight improves the accuracy of alignment.

[0072] Major equipment such as compressors, motors, and control panels are likely to be placed according to the CAD drawings, but equipment such as piping, valves, and meters may be placed in locations that differ from those shown on the CAD drawings depending on the conditions of the installation site.

[0073] Since some equipment cannot be aligned with the CAD drawing in the first place, it is important to give a higher weight to the alignment points specified for equipment that is likely to be positioned according to the CAD drawing.

[0074] As shown in Fig. 5, the designation of three corresponding alignment points is also accepted for the reference point group. Next, the reference point group and the CAD point group are aligned (S7).

[0075] Specifically, the correspondence (coordinate transformation) between the reference point group and the CAD point group is estimated from the correspondence of the above three points.

[0076] Using this estimated coordinate transformation as the initial value, the measurement point cloud and the BIM point cloud are aligned using the ICP algorithm (Iterative Closest Point Algorithm).By aligning the points, coordinate transformation matrix information 15 between the reference point cloud and the CAD point cloud is obtained.

[0077] This example explains an example of alignment using the ICP algorithm, but if attribute information indicating the original data from which the reference point cloud information 14 was created is added to the reference point cloud information 14, it will be possible to select an appropriate alignment method based on the attribute information.

[0078] The coordinate transformation matrix information 15 is expressed as a 4x4 matrix composed of rotations and translations. The coordinate transformation matrices between the reference point group and the CAD point group and between the reference point group are stored in the auxiliary storage device 7 as reference point group information 14, CAD point group information 13, and coordinate transformation matrix information 15 between the CAD point group and the reference point group (S8).

[0079] This processing is executed as preprocessing for display and only needs to be executed once.

[0080] FIG. 10 is an example of a flowchart showing the processing of the measurement point cloud alignment unit in the embodiment of the present invention.

[0081] First, the reference point group, the CAD point group, and the coordinate transformation matrix between the reference point group and the CAD point group are read from the reference point group information 14, the CAD point group information 13, and the coordinate transformation matrix information 15 between the CAD point group information and the reference point group information (S11).

[0082] Next, image information of the area around the pipe captured by camera 5 is read (S12). The camera may be an RGB camera or an RGBD camera. From the read image information, a 3D reconstruction method is used to perform 3D reconstruction of the area around the pipe, and the result is stored in measurement point cloud information 16 (S13).

[0083] An example of the display of the measurement point cloud information 16 is shown in FIG.

[0084] The reference point group is also shown in Figure 6. In Figure 6, the measurement point group is 601 and the reference point group is 602.

[0085] As shown in Figure 6, the reference point cloud and the measurement point cloud have almost the same shape, but their positions and directions are different. Therefore, the reference point cloud and the measurement point cloud are aligned (S14).

[0086] Here, a global registration method can be applied to align the reference point cloud and the measurement point cloud. For example, it is possible to perform global registration using the well-known FPFH feature.

[0087] Global registration does not require interactive processing such as specifying three corresponding points. Furthermore, after global registration, the registration can be refined using the ICP method.

[0088] This alignment process determines a coordinate transformation matrix between the measurement point group and the reference point group.

[0089] In this example, alignment is performed using FPFH features, but if attribute information indicating the original data from which the reference point cloud information 14 was created is added to the reference point cloud information 14, an appropriate alignment method can be selected based on the attribute information.

[0090] Next, a transformation matrix synthesis process (S15) is performed to determine a coordinate transformation matrix 18 between the CAD point group and the measurement point group.

[0091] A coordinate transformation matrix 18 between the CAD point group and the measurement point group is obtained by multiplying the coordinate transformation matrix between the CAD point group and the reference point group by a coordinate transformation matrix 17 between the measurement point group and the reference point group.

[0092] By using this coordinate transformation matrix 18 between the CAD point group and the measurement point group, it is possible to align the measurement point group with the CAD data.

[0093] An example of the display result of aligning the CAD piping image obtained from the CAD data with the measurement point cloud is shown in Figure 7. Next, using a coordinate transformation matrix 18 between the CAD point cloud and the measurement point cloud, display processing unit 19 superimposes the CAD piping image obtained from the CAD data onto the video of the piping at the site on display unit 4. The result of the superimposition display is shown in Figure 8.

[0094] The superimposed display makes it possible to determine whether the piping installation is appropriate. In addition, as shown in Figure 3, pins related to the inspection object can be superimposed to make it possible to determine whether the installation is appropriate. In this way, piping line checks using augmented reality technology have been realized.

[0095] The alignment between the CAD point cloud and the measurement data is composed of two processing units: a reference point cloud alignment unit 11 and a measurement point cloud alignment unit 12. The processing of the reference point cloud alignment unit 11 requires interactive processing of specifying corresponding points.

[0096] However, the processing of the measurement point cloud alignment unit 12 performs global alignment, so interactive alignment is not necessary. Therefore, if the reference point cloud alignment unit 11 is running, the processing of the measurement point cloud alignment unit 12 can perform alignment without interactive processing.

[0097] In piping line checks, the same reference point group can often be used for multiple line check targets, so the processing of the reference point group alignment unit 11 can be shared, improving the efficiency of alignment and the efficiency of piping line checks.

[0098] According to the piping display system of this embodiment, it is possible to check and record whether the piping has been installed correctly when the piping is installed.

[0099] According to the piping display system of this embodiment, once the CAD point cloud and the on-site piping measurement point cloud are aligned during a piping line check, alignment can be easily performed by global alignment during piping line checks, enabling superimposed display. Therefore, it is possible to check and record whether the piping is being installed correctly using augmented reality during piping installation.

[0100] The present invention is not limited to the above-described embodiment, but includes various modifications. The above-described embodiment has been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to an embodiment having all of the described configurations.

[0101] In addition, part of the configuration of one embodiment can be replaced with the configuration of another embodiment, the configuration of another embodiment can be added to the configuration of one embodiment, and part of the configuration of each embodiment can be added, deleted, or replaced with another configuration.

[0102] The above-mentioned configurations, functions, processing units, processing means, etc. may be partly or entirely realized in hardware by designing them as integrated circuits, for example.

[0103] Furthermore, each of the above configurations, functions, etc. may be realized by software, in which a processor interprets and executes a program that realizes each function. Information such as the program, table, and file that realizes each function can be stored in a recording medium such as a memory, a hard disk, a recording device such as an SSD (Solid State Drive), an IC card, an SD card, or a DVD. [Explanation of symbols]

[0104] 1. Piping display system 2...CPU 3...Input section 4...Display section 5. Camera 6...Memory 7…Auxiliary storage device 8...Bus 11...Reference point group alignment unit 12...Measurement point cloud alignment unit 13...CAD point cloud information 14...Reference point cloud information 15... Coordinate transformation matrix between CAD point cloud and reference point cloud 16...Measurement point cloud information 17... Coordinate transformation matrix between reference point group and measurement point group 18... Coordinate transformation matrix between CAD point cloud and measurement point cloud 19...Display processing unit 21...CAD data 22...Plumbing check information

Claims

1. In a piping display system that displays a CAD piping image created from CAD data of the piping on a photographed image, a display unit that displays the photographed image and the CAD piping image on a screen; an input unit that accepts designation of alignment points from the reference point cloud and CAD point cloud displayed on the screen; a reference point cloud alignment unit that aligns the reference point cloud with the CAD point cloud using the received alignment points to obtain a first coordinate transformation matrix that is a coordinate transformation matrix between the CAD point cloud and the reference point cloud; a measurement point cloud alignment unit that determines a measurement point cloud from the captured image, aligns the measurement point cloud with the reference point cloud, determines a second coordinate transformation matrix that is the coordinate transformation matrix between the measurement point cloud and the reference point cloud, and determines a third coordinate transformation matrix that is the coordinate transformation matrix between the CAD point cloud and the measurement point cloud obtained by multiplying the first coordinate transformation matrix by the second coordinate transformation matrix; A piping display system including a display processing unit that uses a third coordinate transformation matrix to superimpose and display CAD data on a captured image.

2. The piping display system according to claim 1, The input unit of the piping display system accepts the designation of three alignment points.

3. The piping display system according to claim 1, The reference point group alignment unit changes the priority of using alignment points for alignment based on weights associated with the alignment points.

4. The piping display system according to claim 1, A piping display system in which the measurement point cloud alignment unit uses the alignment points received by the input unit when calculating the second coordinate transformation matrix.

5. The piping display system according to claim 1, A piping display system in which the measurement point cloud alignment unit performs global alignment when calculating the second coordinate transformation matrix.

6. The piping display system according to claim 1, The reference point cloud alignment unit aligns the reference point cloud and the CAD point cloud based on attribute information added to the reference point cloud.

7. The piping display system according to claim 1, The measurement point cloud alignment unit aligns the measurement point cloud and the reference point cloud based on attribute information added to the reference point cloud.

8. A piping display method for superimposing a CAD piping image created from CAD data of the piping on a photographed image, The display unit displays the captured image and the CAD piping image on the screen. The input unit accepts the specification of alignment points from the reference point cloud and CAD point cloud displayed on the screen, a reference point cloud alignment unit aligns the reference point cloud and the CAD point cloud using the alignment points received, and calculates a first coordinate transformation matrix which is a coordinate transformation matrix between the CAD point cloud and the reference point cloud; a measurement point cloud alignment unit obtains a measurement point cloud from the piping image, aligns the measurement point cloud with the reference point cloud, obtains a second coordinate transformation matrix that is the coordinate transformation matrix between the measurement point cloud and the reference point cloud, and obtains a third coordinate transformation matrix that is the coordinate transformation matrix between the CAD point cloud and the measurement point cloud obtained by multiplying the first coordinate transformation matrix by the second coordinate transformation matrix; A piping display method in which a display processing unit uses a third coordinate transformation matrix to superimpose and display CAD data on a captured image.

Images