Measurement system and recording medium recording a measurement program

Abstract

A measurement system of one mode includes a first calculation unit, an extraction unit, a second calculation unit, and a display control unit. The first calculation unit calculates first information indicating the position and orientation of a camera based on a marker provided on a measurement object, and the camera is configured to measure the depth up to each point of the measurement object, that is, depth information, together with an image of the measurement object. The extraction unit extracts second three-dimensional shape information corresponding to the imaging range of the camera from first three-dimensional shape information representing the three-dimensional shape of the measurement object based on the first information. The second calculation unit compares the depth information with the second three-dimensional shape information, and calculates second information indicating the position and orientation of the camera based on the comparison result between the depth information and the second three-dimensional shape information. The display control unit displays information related to the comparison result between the second three-dimensional shape information and the depth information on a display device based on the second information.

Description

Technical Field

[0001] The embodiment relates to a measurement system and a recording medium storing a measurement program. Background Art

[0002] As one of the methods for matching data of two point groups related to the same measurement object, the ICP (Iterative Closest Point) method is known. Matching methods such as the ICP method can be applied, for example, to the comparison process of images for confirming that the assembly of components is correctly performed.

[0003] Here, in matching methods such as the ICP method, when the number of point groups is large, the processing time tends to increase. Therefore, if a matching method such as the ICP method is simply applied to a parts assembly system, the real-time performance during measurement may be impaired. Summary of the Invention

[0004] A measurement system according to one aspect includes a first calculation unit, an extraction unit, a second calculation unit, and a display control unit. The first calculation unit calculates first information indicating the position and orientation of a camera based on a marker provided on a measurement object, and the camera is configured to measure the depth to each point of the measurement object, that is, depth information, together with an image of the measurement object. The extraction unit extracts second three-dimensional shape information corresponding to the imaging range of the camera from first three-dimensional shape information representing the three-dimensional shape of the measurement object based on the first information. The second calculation unit compares the depth information with the second three-dimensional shape information, and calculates second information indicating the position and orientation of the camera based on the comparison result between the depth information and the second three-dimensional shape information. The display control unit displays information related to the comparison result between the second three-dimensional shape information and the depth information on a display device based on the second information. Brief Description of the Drawings

[0005] Figure 1 is a block diagram showing an example of the configuration of the measurement system according to the first embodiment.

[0006] Figure 2 is a diagram showing an example of the hardware configuration of the measurement system.

[0007] Figure 3 is a flowchart showing the operation of the measurement system according to the first embodiment.

[0008] Figure 4 is a diagram showing the concept of the intersection area.

[0009] Figure 5A is a diagram showing an example of the display process.

[0010] Figure 5B is a diagram showing an example of the display process.

[0011] Figure 5C It is a diagram showing an example of display processing.

[0012] Figure 6 It is a block diagram showing an example of the structure of the measurement system according to the second embodiment.

[0013] Figure 7 It is a flowchart showing the operation of the measurement system according to the second embodiment.

[0014] Figure 8 It is a flowchart showing the guidance process.

[0015] Figure 9A It is a diagram showing an example of the display of a three-dimensional object.

[0016] Figure 9B It is a diagram showing an example of the display of a three-dimensional object. Detailed Embodiments

[0017] Hereinafter, the embodiments will be described with reference to the accompanying drawings.

[0018] [First Embodiment]

[0019] Figure 1 It is a block diagram showing an example of the structure of the measurement system according to the first embodiment. Figure 1 The measurement system 1 shown can be used for measurement in a component assembly system. The measurement object of the measurement system 1 is, for example, a component p assembled in a device D. The component p as the measurement object is mechanically assembled in the device D through an assembly system, for example. The structure of the assembly system is not particularly limited. The component p can also be assembled in the device D by a person, for example.

[0020] The measurement system 1 in the embodiment compares the information on the three-dimensional shape of the device D measured by the camera 2 with the information on the three-dimensional shape of the device D prepared in advance, and presents the comparison result to the user. The user is, for example, an operator who confirms whether the component p has been correctly assembled in the device D.

[0021] As Figure 1 shown, the measurement system 1 includes a first calculation unit 11, an extraction unit 12, a shape database (DB) 13, a second calculation unit 14, and a display control unit 15. The measurement system 1 is configured to be able to communicate with the camera 2. The communication between the measurement system 1 and the camera 2 can be performed wirelessly or wiredly. In addition, the measurement system 1 is configured to be able to communicate with the display device 3. The communication between the measurement system 1 and the display device 3 can be performed wirelessly or wiredly.

[0022] The camera 2 is, for example, held by the user and configured to measure the depth information of the measurement object together with the image of the measurement object. The depth information is information on the distance from the camera 2 to each point on the surface of the device D. Here, the measurement of the depth information based on the camera 2 can be performed, for example, by projecting and receiving infrared light for a second purpose. However, the measurement of the depth information is not limited to this. The depth information can be measured, for example, by the LiDAR (Light Detecting and Ranging) method. In addition, the camera 2 can also be an RGB-D camera, which is a camera configured to be able to measure an RGB-D image. An RGB-D image is an image having a depth image and an RGB color image. A depth image is an image having the depth of each point of the measurement object as the value of a pixel. An RGB color image is an image having the RGB values of each point of the measurement object as the value of a pixel. The camera 2 does not necessarily have to be an RGB color image and can also be a camera capable of measuring a gray scale image.

[0023] The display device 3 is a display device such as a liquid crystal display and an organic EL display. The display device 3 displays various images based on the data transmitted from the measurement system 1.

[0024] The first calculation unit 11 calculates first information indicating the position and orientation of the camera 2 that has photographed the device D based on the marker M pre-configured on the device D. The marker M is a marker with a known size configured at a predetermined position on the device D with a predetermined orientation. For example, the marker M is configured at a predetermined position on the device D such that two mutually orthogonal sides are parallel to the predetermined X-axis and Y-axis in the plane of the device D, and the normal line is parallel to the predetermined Z-axis in the plane of the device D. The marker M is, for example, an AR (Augmented Reality) marker that can be recognized from the image obtained by the camera 2. Two or more markers M can also be configured on one surface of the device D. In addition, the marker M can also be configured on two or more surfaces of the device D.

[0025] The extraction unit 12 extracts, based on the first information, the three-dimensional shape information of the measurement object stored in the shape DB 13 that corresponds to the imaging range of the depth information of the camera 2. As will be described later, the imaging range of the depth information is a pyramidal range centered on the camera 2.

[0026] The shape DB13 stores known three-dimensional shape information of the measurement object. The known three-dimensional shape information can be design drawing data of 3D CAD (Computer Aided Design) of the device D including the measurement object, etc. The known three-dimensional shape information is not limited to design drawing data, and can also be data of any point group or data that can be converted into data of a point group. In addition, the shape DB13 can also be provided outside the measurement system 1. In this case, the extraction unit 12 of the measurement system 1 obtains information from the shape DB13 as needed. In addition, the known three-dimensional shape information may not be registered in the shape DB13, but may be input by the user to the measurement system 1.

[0027] The second calculation unit 14 compares the depth information measured by the camera 2 with the three-dimensional shape information extracted by the extraction unit 12. Specifically, the second calculation unit 14 compares the data of the measurement point group generated from the depth information with the data of the point group constituting the three-dimensional shape information, and performs matching of the data of the two point groups, thereby calculating the second information, which represents the position and posture of the camera 2 with higher accuracy than the first information. The matching of the data of the point group can be implemented using methods such as the ICP (Iterative Closest Point) method and the CPD (Coherent Point Drift) method.

[0028] The display control unit 15 displays information related to the comparison result of the shapes in the second calculation unit 14 on the display device 3 based on the second information. Information related to the comparison result of the shapes is, for example, an image obtained by overlapping an image based on the point group of the measurement object stored in the shape DB13 on the image based on the point group measured by the camera 2. The display control unit 15, for example, based on the position and posture of the high-precision camera 2, establishes correspondence between the point group obtained from the depth information measured by the camera 2 and the image measured by the camera 2, and generates a three-dimensional model of the measurement object. Then, the display control unit 15 overlaps the three-dimensional model based on the known three-dimensional shape information on the generated three-dimensional model of the measurement object and displays it on the display device 3.

[0029] Figure 2 It is a diagram showing an example of the hardware structure of the measurement system 1. The measurement system 1 can be various terminal devices such as a personal computer (PC) and a tablet terminal. As Figure 2 shown, the measurement system 1 has a processor 101, a ROM 102, a RAM 103, a memory 104, an input interface 105, and a communication device 106 as hardware.

[0030] The processor 101 is a processor that controls the overall operation of the measurement system 1. The processor 101 operates as the first calculation unit 11, the extraction unit 12, the second calculation unit 14, and the display control unit 15, for example, by executing a program stored in the memory 104. The processor 101 is, for example, a CPU (Central Processing Unit), and the processor 101 can also be an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like. The processor 101 can be a single CPU or the like, or can be multiple CPUs or the like.

[0031] The read-only memory (ROM) 102 is a non-volatile memory. The ROM 102 stores the startup program of the measurement system 1 and the like. The RAM (Random Access Memory) 103 is a volatile memory. The RAM 103 is used as a working memory during processing in the processor 101, for example.

[0032] The memory 104 is a memory such as a hard disk drive or a solid state drive, for example. The memory 104 stores various programs executed by the processor 101 such as measurement programs. In addition, the memory 104 can store the shape DB 13. The shape DB 13 does not necessarily have to be stored in the memory 104.

[0033] The input interface 105 includes input devices such as a touch panel, a keyboard, and a mouse. When an operation of the input device of the input interface 105 is performed, a signal corresponding to the operation content is input to the processor 101. The processor 101 performs various processes based on this signal.

[0034] The communication device 106 is a communication device used for the measurement system 1 to communicate with external devices such as the camera 2 and the display device 3. The communication device 106 can be a communication device for wired communication or a communication device for wireless communication.

[0035] Next, the operation of the measurement system 1 in the first embodiment will be described. Figure 3 It is a flowchart showing the operation of the measurement system 1 in the first embodiment. Figure 3The processing is executed by the processor 101. Hereinafter, an example will be described in which the camera 2 is an RGB-D camera and the known three-dimensional shape information is the 3D CAD data of the device D. However, as described above, the camera 2 may not be an RGB-D camera, and the known three-dimensional shape information may not be the 3D CAD data of the device D.

[0036] In step S1, the processor 101 acquires an RGB-D image of the device D including the component p to be measured from the camera 2.

[0037] In step S2, the processor 101 detects the marker M based on the color image acquired from the camera 2. The processor 101 transforms the color image acquired from the camera 2 into, for example, a grayscale image, further transforms the grayscale image into a binary black-and-white image, and compares the binary black-and-white image with the pattern of the marker M stored in advance, thereby detecting the marker M. The method for detecting the marker M is not limited to this.

[0038] In step S3, the processor 101 calculates the position and orientation of the camera 2. The processor 101 calculates the position and orientation of the camera 2 based on the position, size, and orientation of the marker M detected in the color image. The marker M is arranged at a predetermined position of the device D in a predetermined orientation. Based on the position, size, and orientation of the marker M in the color image, the position and orientation of the camera 2 relative to the device D can be calculated. For example, the distance from the camera 2 to the marker M, that is, the position of the camera 2, can be calculated based on the position and size of the marker M in the color image. In addition, the inclination of the camera 2 relative to the device D, that is, the orientation of the camera 2, can be calculated based on the inclination of each side (each axis) of the marker M in the color image.

[0039] In step S4, the processor 101 performs coordinate transformation of the 3D CAD data. Specifically, the processor 101 acquires, for example, the 3D CAD data of the device D from the memory 104. Then, the processor 101 transforms the coordinate values of each point in the 3D CAD data into the values in the coordinate system of the camera 2 based on the calculated position and orientation of the camera 2. For example, the processor 101 applies the transformation matrix calculated based on the position and orientation of the camera 2 to the coordinate values of each point in the 3D CAD data, thereby translating and rotating the coordinates of each point in the 3D CAD data.

[0040] In step S5, the processor 101 determines the intersection area between the point group in the 3D CAD data and the imaging range of the camera 2. Figure 4 is a diagram showing the concept of the intersection area. When the depth direction is also considered, the imaging range of the camera 2 is represented by a quadrangular pyramid-shaped range formed around the optical axis of the camera 2. For example, in Figure 4When the camera 2 is located at the position of point C, the imaging range r of the camera 2 is a pyramidal range with point C as the vertex and the optical axis of the camera 2 as the perpendicular line. The intersection area is the overlapping area between the pyramidal imaging range r and the point group constituting the 3D CAD data. Here, if the position and orientation of the camera 2 are known, and the field of view angle or focal length at the time of shooting of the camera 2 is known, the pyramidal imaging range r can be represented by values in the coordinate system of the camera 2. In step S5, the processor 101 determines the range including the coordinate values of the point group of the 3D CAD data in the pyramidal imaging range r as the intersection area. Here, considering the calculation error of the position and orientation of the camera 2, the intersection area may further have a small marginal area.

[0041] In step S6, the processor 101 extracts the 3D CAD data included in the intersection area in the 3D CAD data. The camera 2 measures the depth information within the imaging range. Therefore, the data of the measurement point group generated based on the depth information is also limited to the data within the imaging range of the camera 2. Therefore, it is sufficient for the 3D CAD data to have the data within the imaging range of the camera 2. In the embodiment, in order to shorten the processing time of the point group matching, the 3D CAD data is limited according to the imaging range of the camera 2.

[0042] In step S7, the processor 101 matches the data of the point group constituting the extracted 3D CAD data with the data of the measurement point group generated based on the depth information to calculate the high-precision position and orientation of the camera 2. The data of the measurement point group can be generated by synthesizing after aligning the depth information and the color image data using the ICP method, CPD method, etc. In the embodiment, since the number of the point group of the 3D CAD data is limited according to the imaging range, it is expected that the matching will be completed in a short time.

[0043] In step S8, the processor 101 overlays and displays the three-dimensional image of the measurement object based on the depth information measured by the camera 2 and the three-dimensional image of the measurement object based on the 3D CAD data on the display device 3. Then, the processor 101 ends Figure 3 the processing.

[0044] Figure 5A 、 Figure 5B 、 Figure 5C is a diagram showing an example of the display process in step S8. Here, Figure 5A shows an example of the image of the measurement object based on the 3D CAD data. In addition, Figure 5B shows an example of the image of the measurement object based on the depth information measured by the camera 2. In addition, Figure 5C shows an example of the image actually displayed in step S8. Figure 5AThe image is generated, for example, by pasting a texture on 3D CAD data. Additionally, Figure 5B The image is generated, for example, by pasting a texture or color image data on the data of the measurement point group. Figure 5C The image can be generated, for example, by Figure 5A overlaying the image of Figure 5B and emphasizing the parts with differences. Figure 5A The alignment of the image of Figure 5B with the image of

[0045] Figure 5A can be performed based on the matching result in step S8. Additionally, the highlighting display can be performed by various methods such as changing the color of the parts with differences, adding shades corresponding to the differences, and displaying a frame indicating the parts with differences. Figure 5B As shown, in the image measured after assembly, the part p is not bolted. Therefore, as Figure 5C shown, in the image representing the comparison result, the part p is colored and displayed. By observing the Figure 5C image, the user can identify that the assembly of the part p has not been correctly performed.

[0046] Here, in addition to the Figure 3 processing shown, the depth information and color image measured by the camera 2 can be stored in the memory 104. Such depth information and color image can also serve as evidence for the confirmation operation of part assembly.

[0047] As described above, according to the first embodiment, the point group in the 3D shape information to be compared for the measurement point group generated based on the depth information of the camera 2 is restricted according to the imaging range of the camera 2. Since the point group in the range that cannot be measured by the camera 2 is not required for matching, by restricting the point group of the 3D shape information in advance, the processing time of matching can be shortened. Therefore, the measurement system 1 of the first embodiment can also handle real-time processing.

[0048] Additionally, there may be a case where the density of the measurement point group measured by the camera 2 is different from the density of the point group of the known 3D shape information. In this case, if the ranges of the two point groups to be compared are different, there is a possibility that the matching at the appropriate position is not performed according to the feature quantity. In the first embodiment, since the point group to be compared is restricted so that the ranges of the measurement point group and the point group to be compared are equal, the suppression of the matching error is also achieved.

[0049] [Second Embodiment]

[0050] Next, the second embodiment will be described. Figure 6This is a block diagram showing an example of the structure of the measurement system according to the second embodiment. Here, in Figure 6 for elements that are the same as Figure 1 the same reference signs are assigned. Explanation of elements that are the same as Figure 1 this is appropriately omitted or simplified. Figure 1

[0051] Figure 6 The measurement system 1 shown in

[0052] also has a guiding unit 16. The guiding unit 16 receives the first information calculated by the first calculation unit 11. And the guiding unit 16 performs a process for guiding the user so that the position and posture of the camera 2 become a state suitable for acquiring depth information. This process is, for example, a process of generating an image representing the next measurement object. The image representing the next measurement object can be, for example, a three-dimensional object simulating the measurement object.

[0053] The display control unit 15 of the second embodiment displays the image generated by the guiding unit 16 on the display device 3.

[0053] The hardware structure of the measurement system 1 in the second embodiment can be basically the same as Figure 2 this. In the second embodiment, the processor 101 can also act as the guiding unit 16.

[0054] Next, the operation of the measurement system 1 in the second embodiment will be described. Figure 7 This is a flowchart showing the operation of the measurement system 1 in the second embodiment. Figure 7 The process of Figure 7 is executed by the processor 101. Here, in Figure 3 for processes that are the same as Figure 3 the same reference signs are assigned. Explanation of processes assigned the same reference signs as Figure 3 this is appropriately omitted or simplified.

[0055] The processes of steps S1 - S4 are the same as Figure 3 this. In step S4, in step S11 after the coordinate transformation of the 3D CAD data, the processor 101 performs a guiding process. After the guiding process, the process proceeds to step S12. The guiding process is a process of displaying a three-dimensional object for guiding the user on the display device 3. Hereinafter, the guiding process will be described. Figure 8 This is a flowchart showing the guiding process.

[0056] In step S21, the processor 101 generates a three-dimensional target object for the next measurement object to be guided. The three-dimensional target object is a three-dimensional model simulating the shape of the measurement object. For example, when the measurement object is a component p assembled to the device D, the three-dimensional target object can be a three-dimensional model simulating the shape of the component p. Such a three-dimensional target object can be stored in the memory 104 in advance, for example. In this case, the processor 101 obtains the three-dimensional target object corresponding to the next measurement object from the memory 104.

[0057] In step S22, the processor 101 overlaps the three-dimensional target object of the next measurement object on, for example, the color image of the device D measured by the camera 2. The overlapping position of the three-dimensional target object is the position of the next measurement object in the color image of the device D. Through the processing of step S3, the position and orientation of the camera 2 with respect to the device D based on the marker M are calculated. Based on the position and orientation of the camera 2, the position of the next measurement object in the color image can be determined. Here, preferably, the three-dimensional target object overlapping the color image is rotated according to the orientation of the camera 2 with respect to the device D. In addition, in step S22, the three-dimensional target object of the next measurement object overlaps the color image measured by the camera 2. However, it is not limited thereto. The three-dimensional target object of the next measurement object may overlap the depth image measured by the camera 2 or the 3D CAD data stored in the memory 104. That is, the three-dimensional target object can overlap any image where the user can recognize the position and shooting direction of the next measurement object.

[0058] In step S23, the processor 101 displays the color image of the device D with the three-dimensional target object overlapped thereon on the display device 3. After that, the processor 101 ends Figure 8 the processing and returns the processing to Figure 7 .

[0059] Figure 9A And Figure 9B are diagrams showing display examples of the three-dimensional target object. As Figure 9A And Figure 9B shown, the three-dimensional target object O overlaps the position of the component pn of the next measurement object. By observing Figure 9A And Figure 9B the images, the user can recognize where the position of the next measurement object is and from where to shoot the next measurement object. For example, when the user observes Figure 9A the image, the three-dimensional target object O overlaps with other components, so it is recognized that the component pn of the next measurement object is blocked by other components. In this case, the user can study changing the shooting direction of the camera 2 to shoot Figure 9B such an image.

[0060] Here, in addition to Figure 8In addition to the processing of , a color image with a 3D object superimposed thereon can also be stored in the memory 104. Such a color image can also serve as evidence for the confirmation operation of component assembly.

[0061] Here, a description of Figure 7 is given. In step S12 after the guidance process, the processor 101 determines whether the imaging range of the camera 2 is appropriate. In step S12, when it is determined that the imaging range of the camera 2 is appropriate, the process proceeds to step S5. In step S12, when it is determined that the imaging range of the camera 2 is inappropriate, the process returns to step S1. In this case, the user changes the shooting direction of the camera 2 while observing the image displayed on the display device 3 and performs shooting of the component to be measured again.

[0062] A description of the determination in step S12 is given. Whether the imaging range of the camera 2 is appropriate is determined based on whether it is an imaging range for obtaining a sufficient point group related to the next measurement object. For example, in Figure 9A , the component pn as the next measurement object is hidden by other components. For the part hidden by other components, a point group cannot be obtained. In such a case, it is determined that the imaging range is inappropriate. Therefore, for example, in step S12, when the overlapping range between the component p to be measured and the 3D object O is below the threshold, it can be determined that the imaging range is appropriate. In addition, in step S12, in addition to the determination of whether the overlapping range is below the threshold, it can also be determined that the imaging range is appropriate when the size in the color image of the component p to be measured is above the threshold.

[0063] In step S12, after the process proceeds to step S5, the processor 101 performs processing in the same manner as in the first embodiment. The description of the processing in steps S5 - S8 is omitted.

[0064] As described above, in the second embodiment, based on the position and orientation of the camera 2 with respect to the device D with the marker M as a reference, processing is performed to guide the user to make the position and orientation of the camera 2 in a state suitable for obtaining depth information. Thereby, shooting is performed in an appropriate position and orientation, and as a result, the error in matching the measured point group with the point group in the known 3D shape information can be suppressed.

[0065] In addition, in the second embodiment, the 3D object representing the component as the next measurement object is, for example, superimposed and displayed on the color image. Thereby, the user can be guided to perform the confirmation operation of component assembly in a predetermined order.

[0066] [Modification Example]

[0067] A modification example of the first embodiment and the second embodiment will be described. In the first embodiment and the second embodiment, the measurement system 1 is used for measurement in a component assembly system. In contrast, the measurement systems of the first embodiment and the second embodiment can be applied to any measurement system that performs matching between first point group data based on depth information measured by the camera 2 and second point group data scattered within a range wider than the first point group data. In this case, the processor 101 extracts point group data of the intersection area with the imaging range of the camera 2 from the second point group data.

[0068] In addition, in the first embodiment and the second embodiment, the marker M is set as an AR marker. In contrast, if it is a marker that can calculate the position and orientation of the camera 2, it is not necessary to use an AR marker accompanied by image recognition. For example, as the marker M, other markers such as a light marker can also be used. A light marker is a marker that is recognized by a combination of a light projection element and a light receiving element. By arranging three or more groups of light markers on the surface of the device D, the position and orientation of the camera 2 relative to the device D can be calculated. In addition, a 2D code, a barcode, a calibration plate, etc. can also be used as the marker M.

[0069] In addition, in the first embodiment and the second embodiment, the camera 2 can also be integrally formed with the measurement system 1. In this case, the control of the position and orientation of the camera 2 can also be implemented by the measurement system 1.

[0070] Several embodiments of the present invention have been described, but these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other ways, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope or gist of the invention, and are also included in the scope of the invention described in the claims and its equivalents.

Claims

1. A measurement system, comprising: A first calculation unit that calculates first information representing the position and orientation of a camera based on a marker provided on a measurement object, the camera being configured to measure the depth to each point of the measurement object, i.e., depth information, together with an image of the measurement object; An extraction unit that extracts second three-dimensional shape information corresponding to the imaging range of the camera from first three-dimensional shape information based on the first information, the first three-dimensional shape information representing the three-dimensional shape of the measurement object; A second calculation unit that compares the depth information with the second three-dimensional shape information and calculates second information representing the position and orientation of the camera with higher accuracy than the first information based on the comparison result between the depth information and the second three-dimensional shape information; And A display control unit that displays information regarding the comparison result between the second three-dimensional shape information and the depth information on a display device based on the second information.

2. The measurement system according to claim 1, Wherein, It further includes a guiding unit that guides the user of the camera based on the first information so that the position and orientation of the camera are in a state suitable for obtaining the depth information.

3. The measurement system according to claim 2, Wherein, The guiding unit guides the user of the camera by displaying a target object representing the measurement object on the display device.

4. The measurement system according to any one of claims 1 to 3, Wherein, The marker is an AR marker, i.e., an augmented reality marker, The extraction unit detects the AR marker by image recognition based on the image of the measurement object obtained by the camera.

5. The measurement system according to claim 4, Wherein, The image of the measurement object is a color image of the measurement object.

6. A recording medium that records a measurement program for causing a computer to execute the following processing: Calculating first information representing the position and orientation of a camera based on a marker provided on a measurement object, the camera being configured to measure the depth to each point of the measurement object, i.e., depth information, together with an image of the measurement object; Extracting second three-dimensional shape information corresponding to the imaging range of the camera from first three-dimensional shape information based on the first information, the first three-dimensional shape information representing the three-dimensional shape of the measurement object; Comparing the depth information with the second three-dimensional shape information and calculating second information representing the position and orientation of the camera with higher accuracy than the first information based on the comparison result between the depth information and the second three-dimensional shape information; And Displaying information regarding the comparison result between the second three-dimensional shape information and the depth information on a display device based on the second information.

Images