Position and orientation detection device and position and orientation detection method

Abstract

The present invention provides a position and orientation detection device and a position and orientation detection method capable of accurately and easily detecting the position and orientation of a workpiece placed on a rotary table. The position and orientation detection device comprises: a camera (64) fixed in a position separated from a rotary table (60); a display (68) provided in a position facing the camera (64) across a workpiece (W); a background simulator unit (46) that displays, on the display (68), a background image that changes in accordance with the rotation angle of the rotary table (60); an image acquisition unit (48) that acquires a plurality of captured images captured by the camera (64) at a plurality of positions at which the rotation angle of the rotary table (60) is different from each other; a point cloud information generation unit (50) that generates three-dimensional point cloud information of the workpiece (W) on the basis of the plurality of captured images; and a position and orientation determination unit (52) that determines the position and orientation of the workpiece (W) on the basis of the three-dimensional point cloud information.

Description

Position and orientation detecting apparatus and position and orientation detecting method

[0001] The present invention relates to a position and orientation detecting apparatus and a position and orientation detecting method for detecting the position and orientation of a workpiece on a rotary table.

[0002] Conventionally, a coordinate measuring machine (CMM) that measures the three-dimensional shape of a workpiece by moving a probe relative to the workpiece and bringing the probe into contact with the workpiece is known.

[0003] In addition, there has been known a three-dimensional coordinate measuring machine further provided with a rotary table for the purpose of efficiently measuring rotating system measurement objects (for example, gears, cylindrical cams, impellers, etc.) (see, for example, Patent Document 1). The rotary table is installed on a surface plate of the three-dimensional coordinate measuring machine, is configured to be capable of placing a workpiece thereon and rotating the workpiece about a rotation axis.

[0004] Japanese Patent Application Laid-Open No. 2009-271030

[0005] When measuring a workpiece placed on a rotary table in a three-dimensional coordinate measuring machine, an operator needs to make the three-dimensional coordinate measuring machine recognize the position and orientation of the workpiece, which requires complicated work such as manual probing. Therefore, this results in heavy workload for the operator and time-consuming work.

[0006] To address such problems, for example, by fixing a workpiece installation jig on the rotary table and positioning the workpiece with the jig, the relative positional relationship between the rotary table and the workpiece is uniquely determined. However, in the method using a workpiece installation jig, the jig must be fixed on the rotary table, and a special jig needs to be prepared according to the type of workpiece, which has the disadvantage that the versatility of the three-dimensional coordinate measuring machine is lost. In addition, generally the load capacity of the rotary table is small, so it is not desirable to add the weight of a jig having a function of indexing the position and orientation of the workpiece.

[0007] As another method, a method is also conceivable in which three-dimensional point cloud information representing the surface of a workpiece is acquired using a line laser or the like, best fit matching is performed with the three-dimensional CAD model (design data) of the workpiece, and the position and orientation of the workpiece are obtained. However, line lasers that can generally capture a wide range are susceptible to glossy surfaces, so complicated work is required, such as the need to coat the surface of the workpiece with powder that scatters light.

[0008] The present invention has been made in view of such circumstances, and an object of the present invention is to provide a position and orientation detection apparatus and a position and orientation detection method capable of accurately and easily detecting the position and orientation of a workpiece placed on a rotary table.

[0009] In order to achieve the above object, the present invention is constituted by the following aspects.

[0010] A position and orientation detection apparatus according to a first aspect is a position and orientation detection apparatus that detects the position and orientation of a workpiece placed on a rotary table, comprising: a camera fixed at a position spaced apart from the rotary table; a display provided at a position facing the camera with the workpiece interposed therebetween; a background simulator unit that displays a background image that changes in accordance with the rotation angle of the rotary table on the display; an image acquisition unit that acquires a plurality of captured images captured by the camera at a plurality of positions with different rotation angles of the rotary table; a point cloud information generation unit that generates three-dimensional point cloud information of the workpiece based on the plurality of captured images; and a position and orientation determination unit that determines the position and orientation of the workpiece based on the three-dimensional point cloud information.

[0011] In the position and orientation detection apparatus according to a second aspect, in the first aspect, the point cloud information generation unit generates three-dimensional point cloud information of the workpiece by NeRF or 3D Gaussian Splatting.

[0012] In the position and orientation detection apparatus according to a third aspect, in the first aspect or the second aspect, the background simulator unit rotationally moves a virtual camera arranged in a virtual space based on the rotation angle of the rotary table, and displays, on the display, a background image generated based on an image obtained by capturing a virtual background model in the virtual space with the virtual camera.

[0013] In the position and orientation detection device according to the fourth embodiment, in the third embodiment, the background simulator unit converts the captured image taken by the virtual camera based on the relative positional relationship between the camera and the display, and displays the converted captured image as a background image on the display.

[0014] In the fifth embodiment of the position and orientation detection device, in the first embodiment, the background simulator unit displays a background image on a display based on the captured image taken by another camera when the other rotary table is rotated with another camera mounted on the other rotary table, which is positioned in real space.

[0015] The position and orientation detection device according to the sixth embodiment is a position and orientation detection device for detecting the position and orientation of a workpiece placed on a rotary table, comprising: a camera fixed at a position away from the rotary table; a background material for chroma key compositing provided at a position opposite the camera with the workpiece in between; an image acquisition unit that acquires multiple captured images taken by the camera at multiple positions with different rotation angles of the rotary table; a background simulator unit that generates a background image that changes in accordance with the rotation angle of the rotary table; an image compositing unit that generates multiple composite images by compositing each of the multiple captured images with an image obtained by excluding the portion corresponding to the background material from the captured image taken by the camera and a background image corresponding to the rotation angle of the rotary table; a point cloud information generation unit that generates three-dimensional point cloud information of the workpiece based on the multiple composite images; and a position and orientation determination unit that determines the position and orientation of the workpiece based on the three-dimensional point cloud information.

[0016] In the seventh embodiment, the position and orientation detection device, in any of the first to sixth embodiments, has a rotary table installed on the surface plate of a three-dimensional coordinate measuring machine, and the position and orientation determination unit determines the relative position and orientation of the workpiece with respect to the three-dimensional coordinate measuring machine.

[0017] The position and orientation detection method according to the eighth aspect is a position and orientation detection method for detecting the position and orientation of a workpiece placed on a rotary table, comprising: a display step in which a camera and a display are arranged opposite each other with the workpiece in between, and a background image that changes in accordance with the rotation angle of the rotary table is displayed on the display; an image acquisition step in which a plurality of captured images are obtained by the camera at a plurality of positions where the rotation angles of the rotary table are different from each other; a point cloud information generation step in which three-dimensional point cloud information of the workpiece is generated based on the plurality of captured images; and a determination step in which the position and orientation of the workpiece are determined based on the three-dimensional point cloud information.

[0018] The position and orientation detection method according to the ninth embodiment is a position and orientation detection method for detecting the position and orientation of a workpiece placed on a rotary table, comprising: a background image generation step in which a camera and a background material for chroma key compositing are arranged opposite each other with the workpiece in between, and a background image is generated that changes in accordance with the rotation angle of the rotary table; an image acquisition step in which a plurality of captured images are obtained by the camera at a plurality of positions where the rotation angles of the rotary table are different from each other; an image compositing step in which a plurality of composite images are generated by compositing each of the plurality of captured images with a portion corresponding to the background material removed from the captured image taken by the camera, and a background image corresponding to the rotation angle of the rotary table; a point cloud information generation step in which three-dimensional point cloud information of the workpiece is generated based on the plurality of composite images; and a determination step in which the position and orientation of the workpiece are determined based on the three-dimensional point cloud information.

[0019] According to the present invention, the position and orientation of a workpiece placed on a rotary table can be detected accurately and easily.

[0020] This is a schematic configuration diagram showing a three-dimensional coordinate measuring machine according to the first embodiment. This is a functional block diagram of the computer according to the first embodiment. This is a diagram showing the planar positional relationship between the rotary table and the camera. This is a schematic diagram for explaining the functions of the background simulator unit. This is a flowchart showing the flow of the position and orientation detection process. This is a schematic diagram showing the relative positional relationship between the camera and the rotary table in the three-dimensional coordinate measuring machine. This is a schematic diagram showing the relative positional relationship between the camera and the display in the three-dimensional coordinate measuring machine. This is a diagram for explaining the second embodiment. This is a functional block diagram of the computer in the third embodiment. This is a diagram for explaining the position and orientation detection process in the third embodiment.

[0021] Embodiments of the present invention will be described below with reference to the attached drawings.

[0022] <First Embodiment> [Three-Dimensional Coordinate Measuring Machine] Figure 1 is a schematic diagram showing a three-dimensional coordinate measuring machine 10 according to the first embodiment. The three-dimensional coordinate measuring machine 10 measures the shape of the measurement elements of the workpiece W while displacing the position and orientation of the probe 12a. The XYZ axes in Figure 1 represent a machine coordinate system determined based on the machine coordinate origin unique to the three-dimensional coordinate measuring machine 10.

[0023] As shown in Figure 1, the three-dimensional coordinate measuring machine 10 comprises a base plate 16 mounted on a plurality of legs 14, a right Y carriage 18R and a left Y carriage 18L erected at both ends of the base plate 16, and an X guide 20 connecting the upper parts of the right Y carriage 18R and the left Y carriage 18L. The right Y carriage 18R, the left Y carriage 18L and the X guide 20 constitute a gantry frame 22.

[0024] Sliding surfaces are formed on the upper and side surfaces of both ends of the surface plate 16 in the X-axis direction, on which the right Y carriage 18R and the left Y carriage 18L slide along the Y-axis direction. In addition, air bearings (not shown) are provided on the right Y carriage 18R and the left Y carriage 18L at positions facing the sliding surfaces of the surface plate 16. As a result, the right Y carriage 18R and the left Y carriage 18L can move freely in the Y-axis direction together with the X guide 20.

[0025] An X-carriage 24 is attached to the X-guide 20. The X-guide 20 has a sliding surface formed along the X-axis direction on which the X-carriage 24 slides. In addition, an air bearing (not shown) is provided on the X-carriage 24 at a position opposite to the sliding surface of the X-guide 20. This allows the X-carriage 24 to move freely in the X-axis direction.

[0026] A Z-carriage (also called a Z-spindle) 26 is attached to the X-carriage 24. The X-carriage 24 is also provided with an air bearing (not shown) for Z-axis direction guidance, which guides the Z-carriage 26 in the Z-axis direction. As a result, the Z-carriage 26 is held by the X-carriage 24 so as to be movable in the Z-axis direction. A probe head 12 is attached to the lower end of the Z-carriage 26.

[0027] The probe head 12 holds the base end of the contact-type probe 12a. The base end of the stylus 12b is attached to the tip of the probe 12a. A contact element 12c is attached to the tip of the stylus 12b. The stylus 12b and the contact element 12c constitute the measuring element of the probe 12a. The type of probe 12a is not particularly limited.

[0028] The probe 12a is rotated by the drive unit 32 (see Figure 2) around two mutually orthogonal rotation axes (not shown).

[0029] A drive unit 32 (see Figure 2) is provided, which includes a Y-axis drive unit for moving the gantry frame 22 in the Y-axis direction, an X-axis drive unit for moving the X carriage 24 in the X-axis direction, and a Z-axis drive unit for moving the Z carriage 26 in the Z-axis direction. Each drive unit is composed of a known drive mechanism including a motor. This makes it possible to move the probe head 12 and probe 12a in the three axes of X, Y, and Z.

[0030] A linear scale (not shown) for detecting the Y-axis position is provided at the right Y-carriage 18R side end of the surface plate 16. Additionally, a linear scale (not shown) for detecting the X-axis position is provided on the X-guide 20, and a linear scale (not shown) for detecting the Z-axis position is provided on the Z-carriage 26.

[0031] The right Y carriage 18R is equipped with a Y-axis position detection head (not shown) for reading a linear scale for detecting the Y-axis position. The X carriage 24 is equipped with an X-axis position detection head (not shown) for reading a linear scale for detecting the X-axis position, and a Z-axis position detection head (not shown) for reading a linear scale for detecting the Z-axis position. Furthermore, the probe head 12 is equipped with a rotation angle detection unit (not shown), such as a rotary encoder, for detecting the rotation angle of the probe 12a.

[0032] The three-dimensional coordinate measuring machine 10 detects the coordinates in the XYZ axis direction of each measurement point (such as the inner surface) of the workpiece W when the contact element 12c at the tip of the probe 12a contacts each measurement point (such as the inner surface) of the workpiece W, based on the detection results of the direction position detection heads for each of the XYZ axes and the detection results of the rotation angle detection unit.

[0033] The three-dimensional coordinate measuring machine 10 is equipped with a rotary table 60. The rotary table 60 is provided on the upper surface of the surface plate 16. The rotary table 60 is configured to be able to place a workpiece W on it and to rotate the workpiece W around a rotation axis (R axis). The rotation axis of the rotary table 60 is parallel to the Z-axis direction, but it does not necessarily have to be parallel; for example, it may be tilted at an angle to the Z-axis direction. The rotary table 60 is equipped with a rotation angle detection unit 62 (see Figure 2), such as a rotary encoder, for detecting the rotation angle of the rotary table 60. The output of the rotation angle detection unit 62 (rotation angle of the rotary table 60) is output to the computer 30. The output of the rotation angle detection unit 62 may also be output to the computer 30 via a drive controller 28.

[0034] The three-dimensional coordinate measuring machine 10 also includes a camera 64 and a display 68. The camera 64 is fixed to the upper surface of the base plate 16 at a position away from the rotary table 60 (in this example, on the front side of the three-dimensional coordinate measuring machine 10) via a rod-shaped support member 66. The display 68 is positioned opposite the camera 64, with the workpiece W on the rotary table 60 in between. The camera 64 and the display 68 are used in the position and orientation detection process described later. The camera 64 and the display 68 may be removed from the three-dimensional coordinate measuring machine 10 when not performing the position and orientation detection process (for example, when measuring each measurement element of the workpiece W). Alternatively, the camera 64 and the display 68 may be left permanently attached to the three-dimensional coordinate measuring machine 10 as long as they do not interfere with the measurement of each measurement element of the workpiece W.

[0035] The three-dimensional coordinate measuring machine 10 is equipped with a drive controller 28 that controls the drive unit 32 to control the movement of the probe head 12, that is, the displacement of the position and orientation of the probe 12a (stylus 12b). Here, the three-dimensional coordinate measuring machine 10 has an automatic measurement mode in which measurements are performed automatically and a manual measurement mode in which measurements are performed manually. Therefore, in the automatic measurement mode, the drive controller 28 controls the drive unit 32 under the control of the computer 30 described later to displace the position and orientation of the probe 12a.

[0036] Furthermore, the drive controller 28 is provided with a probe operating unit 28a, such as a joystick, for manually controlling the position and orientation of the probe 12a (stylus 12b). Therefore, in manual measurement mode, the drive controller 28 controls the drive unit 32 in response to the operation input to the probe operating unit 28a, thereby displacing the position and orientation of the probe 12a.

[0037] The drive controller 28 is connected to a contact detection sensor (not shown) of the probe 12a, a direction position detection head (not shown) for each of the XYZ axes (not shown), and a rotation angle detection unit (not shown) of the probe 12a. The moment the contact detection sensor detects that the contact element 12c of the probe 12a has come into contact with a measurement point of the workpiece W, the drive controller 28 acquires the detection results from the direction position detection head and the rotation angle detection unit for each of the XYZ axes, and detects the coordinates of each measurement point in the XYZ axis direction. The coordinates of each measurement point are output from the drive controller 28 to the computer 30.

[0038] The computer 30 is connected to the drive controller 28 via various communication interfaces such as a LAN (Local Area Network) to enable data communication.

[0039] The computer 30 has a software program 30a installed on it. By executing the software program 30a, the computer 30 performs various measurement operations, including acquiring the coordinates of each measurement point.

[0040] [Computer Functions] Figure 2 is a functional block diagram of the computer 30. As shown in Figure 2, the computer 30 includes a control unit 34 that comprehensively controls the operation of each part of the three-dimensional coordinate measuring machine 10.

[0041] The control unit 34 includes an arithmetic circuit composed of various processors and memory. These various processors include CPUs (Central Processing Units), GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), and programmable logic devices [e.g., SPLDs (Simple Programmable Logic Devices), CPLDs (Complex Programmable Logic Devices), and FPGAs (Field Programmable Gate Arrays)]. The various functions of the control unit 34 may be implemented by a single processor, or by multiple processors of the same or different types.

[0042] Furthermore, the control unit 34 includes a storage unit 36 ​​that stores the aforementioned software program 30a. The control unit 34 functions as a drive control unit 40, a shape calculation unit 42, and a position and orientation detection unit 44 by executing the software program 30a in the storage unit 36. In addition to the software program 30a, the storage unit 36 ​​also stores background data 70 and CAD data 72, which will be described later. The CAD data 72 is design data that shows the design shape of the workpiece W. The software program 30a also includes a measurement program that describes the measurement positions and measurement order of the measurement elements.

[0043] The drive control unit 40 operates in the automatic measurement mode described above. Based on the software program 30a (measurement program) in the storage unit 36, the drive control unit 40 drives the drive unit 32 via the drive controller 28, thereby making the probe 12a contact all measurement points for each measurement element of the workpiece W.

[0044] When measuring each measuring element of the workpiece W (in automatic measurement mode and manual measurement mode), the shape calculation unit 42 obtains the coordinate values ​​of the measurement points from the drive controller 28 each time the probe 12a contacts the measurement points of each measuring element of the workpiece W, and calculates the shape of the measuring element based on the coordinate values ​​of all the measurement points of the measuring element. Note that the specific method for calculating the shape of the measuring element is known technology, so a detailed explanation is omitted here.

[0045] The position and orientation detection unit 44 performs processing (hereinafter referred to as "position and orientation detection processing") to detect the relative position and orientation of the workpiece W with respect to the three-dimensional coordinate measuring machine 10. The position and orientation detection unit 44 comprises a background simulator unit 46, an image acquisition unit 48, a point cloud information generation unit 50, and a position and orientation determination unit 52. The position and orientation detection unit 44 is a part of the elements that constitute the position and orientation detection device of the present invention.

[0046] [Position and Attitude Detection Process] First, the general flow of the position and attitude detection process performed by the position and attitude detection unit 44 will be explained with reference to Figure 3. Figure 3 is a diagram showing the planar positional relationship between the rotary table 60 and the camera 64.

[0047] In the position and orientation detection process, a camera 64 fixed at a position away from the rotary table 60 is used to acquire multiple images of the workpiece W on the rotary table 60, taken from multiple different viewpoints.

[0048] Specifically, as shown in Figure 3, the rotary table 60 on which the workpiece W is placed is rotated around the rotation center C1 (rotation axis). Then, each time the rotation angle of the rotary table 60 changes by a predetermined angle, the workpiece W on the rotary table 60 is periodically photographed by the camera 64. The predetermined angle is set appropriately according to the number of images required in the three-dimensional point cloud information generation method described later. For example, if M images are to be acquired during one rotation of the rotary table 60, the predetermined angle is set to a value of 360° / M (where M is a number of 2 or more). As a result, multiple (M) images of the workpiece W on the rotary table 60, taken from multiple viewpoints, are acquired during one rotation of the rotary table 60.

[0049] Furthermore, if multiple images can be obtained of the workpiece W on the rotary table 60 from multiple different viewpoints, the number of rotations of the rotary table 60 is not limited to one but may be multiple. Also, the camera 64 is not limited to taking periodic shots according to the rotation angle of the rotary table 60, but may take aperiodic shots.

[0050] Next, three-dimensional point cloud information of the workpiece W is generated using multiple images captured by the camera 64. As a method for generating the three-dimensional point cloud information, for example, NeRF (Neural Radiance Field) or 3D Gaussian Platting (hereinafter referred to as "3DGS") can be used.

[0051] NeRF and 3DGS are technologies (hereinafter referred to as "three-dimensional reconstruction technologies") that reconstruct three-dimensional models (objects and scenes in three-dimensional space) from images from multiple viewpoints. NeRF learns from images from multiple viewpoints using a neural network and reconstructs a three-dimensional model using a luminance field. 3DGS reconstructs a three-dimensional model using a 3D Gaussian distribution instead of a neural network.

[0052] By utilizing such three-dimensional reconstruction technology, it is possible to generate three-dimensional point cloud information of the workpiece W from multiple images captured by the camera 64. Since NeRF and 3DGS are publicly known, a detailed explanation is omitted here. Furthermore, the method for generating the three-dimensional point cloud information is not limited to NeRF and 3DGS; any method capable of generating three-dimensional point cloud information of the workpiece W from images from multiple viewpoints may be used.

[0053] Next, the position and orientation of the workpiece W on the rotary table 60 are determined by fitting a three-dimensional CAD model of the workpiece W to the three-dimensional point cloud information of the workpiece W generated from multiple captured images. The three-dimensional CAD model of the workpiece W is a model representing the three-dimensional shape of the workpiece W and is generated based on the CAD data 72 of the workpiece W stored in the storage unit 36. This makes it possible for the three-dimensional coordinate measuring machine 10 to recognize the relative positional relationship between the machine coordinate system defined in the three-dimensional coordinate measuring machine 10 and the workpiece coordinate system defined in the workpiece W, and the three-dimensional coordinate measuring machine 10 becomes ready to start measuring each measurement element of the workpiece W.

[0054] The general flow of the position and orientation detection process is as described above.

[0055] Here, we will explain the problems in the position and orientation detection process described above.

[0056] Three-dimensional reconstruction technologies such as NeRF or 3DGS are methods that optimize a three-dimensional model to minimize errors when rendering it relative to an image, and are relatively robust against specular surfaces. On the other hand, when images from multiple viewpoints are taken against a plain background (or in an environment where the background hardly changes), the three-dimensional model for each image cannot be uniquely determined, which can easily lead to a huge amount of noise.

[0057] In the position and orientation detection process described above, the workpiece W is placed on a rotary table 60, which is rotated while a camera 64 fixed at a position away from the rotary table 60 captures images of the workpiece W on the rotary table 60. Therefore, the viewpoint position of the camera 64 relative to the workpiece W changes with each capture, while the background of each captured image does not change or changes very little. Consequently, when attempting to generate three-dimensional point cloud information of the workpiece W from multiple captured images using the three-dimensional reconstruction technique described above, the increase in noise becomes a factor that increases the error in the three-dimensional point cloud information of the workpiece W.

[0058] Therefore, in this embodiment, as shown in Figure 1, a display 68 is positioned opposite the camera 64, with the workpiece W on the rotary table 60 in between. The display 68 displays a background image that changes in accordance with the rotation angle of the rotary table 60. As a result, the background image included in each image captured by the camera 64 changes in accordance with the rotation angle of the rotary table 60, thus resolving the above problem. Note that the camera 64 is an example of the camera of the present invention. The display 68 is an example of the display of the present invention.

[0059] Next, the configuration for generating the background image to be displayed on the display 68 will be described with reference to Figure 4. Figure 4 is a schematic diagram illustrating the functions of the background simulator unit 46.

[0060] As shown in Figure 4, the background simulator unit 46 is an example of the background simulator unit of the present invention, and generates a background image for display on the display 68. Specifically, the background simulator unit 46 generates a background image as follows.

[0061] The background simulator unit 46 acquires background data 70 from the storage unit 36 ​​(see Figure 2) as a preprocessing step for generating a background image to be displayed on the display 68. The background data 70 is data for generating a three-dimensional model (hereinafter referred to as the "virtual background model") 104 that shows the three-dimensional shape of the background in the virtual space (CG space). The background data 70 also includes texture data (background texture) for coloring the virtual background model.

[0062] When the background simulator unit 46 obtains background data 70 from the storage unit 36, it generates a virtual background model 104 in the virtual space based on the background data 70 as part of the virtual background model generation process. The virtual background model 104 is information that shows the background in all directions (360 degrees) centered on the rotation center C2 corresponding to the rotation center C1 of the rotary table 60.

[0063] The background simulator unit 46 generates a virtual background model 104 in the virtual space, and then places a virtual camera 100 and a virtual screen 102 in the virtual space at positions opposite each other with respect to the rotation center C2.

[0064] The virtual camera 100 is equivalent to camera 64. Specifically, the distance from the rotation center C1 to camera 64 and the distance from the rotation center C2 to the virtual camera 100 are set to be equal. Furthermore, the camera parameters (internal parameters and distortion parameters) of the virtual camera 100 and camera 64 are assumed to be equal to each other. Note that the virtual camera 100 is an example of a virtual camera in the present invention.

[0065] Furthermore, the virtual screen 102 is equivalent to the display 68. Specifically, the distance from the rotation center C1 to the display 68 and the distance from the rotation center C2 to the virtual screen 102 are configured to be equal to each other. In addition, the size and shape of the display 68 and the virtual screen 102 are assumed to be equal to each other.

[0066] The background simulator unit 46 receives the rotation angle of the rotary table 60 detected by the rotation angle detection unit 62. Based on the rotation angle of the rotary table 60, the background simulator unit 46 rotates the virtual camera 100 and virtual screen 102 around the rotation center C2. At this time, as shown in Figure 4, the rotation angles of the virtual camera 100 and virtual screen 102 and the rotation angle of the rotary table 60 are the same, but their rotation directions are opposite. That is, the virtual camera 100 and virtual screen 102 are rotated by an angle that has the same absolute value as the rotation angle of the rotary table 60, but with opposite signs. Note that the rotation angles (absolute values) of the virtual camera 100 and virtual screen 102 and the rotation angle (absolute values) of the rotary table 60 do not necessarily have to be exactly equal; some error is acceptable.

[0067] The background simulator unit 46 rotates the virtual camera 100 and virtual screen 102 in the virtual space as described above, and then acquires an image captured by the rotated virtual camera 100 of the virtual background model 104. The background simulator unit 46 then projects the image captured by the virtual camera 100 onto the virtual screen 102 to generate a projected image, and outputs this projected image as the background image to the display 68. As a result, the display 68 displays a background image that changes in accordance with the rotation angle of the rotary table 60.

[0068] Next, the flow of the position and attitude detection process (position and attitude detection method) executed by the position and attitude detection unit 44 will be explained in detail with reference to Figure 5. Figure 5 is a flowchart showing the flow of the position and attitude detection process. Note that the position and attitude detection process is an example of the position detection method of the present invention.

[0069] First, before the position and orientation detection process begins, a camera 64 and a display 68 are installed on the three-dimensional coordinate measuring machine 10. The camera 64 is fixed on the surface plate 16 at an arbitrary position away from the rotary table 60, supported by a rod-shaped support member 66. The display 68 is installed in a position opposite the camera 64, with the workpiece W on the rotary table 60 in between. In Figure 1, as an example, the camera 64 is positioned on the front side (one side in the Y direction) of the surface plate 16 relative to the rotary table 60, and the display 68 is positioned on the rear side (the other side in the Y direction) of the surface plate 16 relative to the rotary table 60. However, the camera 64 and the display 68 may be positioned in other positions as long as they are in a positional relationship where they face each other with the workpiece W on the rotary table 60 in between. The display 68 is also positioned within the field of view of the camera 64.

[0070] When the position and orientation detection process is started, the background simulator unit 46 first acquires background data 70 from the storage unit 36 ​​(step S10). Then, the background simulator unit 46 generates a virtual background model 104 in the virtual space based on the background data 70 acquired from the storage unit 36. It also places a virtual camera 100 and a virtual screen 102 in the virtual space. The virtual camera 100 and the virtual screen 102 are equivalent to the camera 64 and the display 68, respectively, and the specific placement conditions are as described above.

[0071] Next, a workpiece imaging process is performed in which the workpiece W on the rotary table 60 is photographed by the camera 64 while the rotary table 60 is rotating (step S12). The workpiece imaging process includes the processes shown in steps S14 to S22.

[0072] When the workpiece imaging process begins, the rotation angle detection unit 62 first detects the rotation angle of the rotary table 60. The rotation angle of the rotary table 60 detected by the rotation angle detection unit 62 is output to the background simulator unit 46. As a result, the background simulator unit 46 obtains the rotation angle of the rotary table 60 (step S14).

[0073] Next, the background simulator unit 46 generates a background image based on the rotation angle of the rotary table 60 and outputs the background image to the display 68 (step S16).

[0074] Specifically, the background simulator unit 46 rotates the virtual camera 100 in the virtual space by an angle of rotation that is the same in absolute value as the rotation angle of the rotary table 60 but with the sign reversed (i.e., the same rotation angle in the opposite direction), and acquires an image of the virtual background model 104 captured by the virtual camera 100.

[0075] The background simulator unit 46 then projects the image captured by the virtual camera 100 onto the virtual screen 102, generates a projected image, and outputs this projected image as a background image to the display 68. As a result, the display 68 displays a background image that changes in accordance with the rotation angle of the rotary table 60.

[0076] Next, with a background image corresponding to the rotation angle of the rotary table 60 displayed on the display 68, the camera 64 photographs the workpiece W on the rotary table 60 (step S18). The image captured by the camera 64 is output to the computer 30. The image acquisition unit 48 then acquires the image captured by the camera 64.

[0077] Next, a determination is made as to whether the workpiece photography process is complete (step S20). This determination may be made by the determination unit (not shown) of the position and orientation detection unit 44, or it may be made by the user (operator). For example, the determination of whether the workpiece photography process is complete is made as to whether the rotation speed of the rotary table 60 or the number of images captured by the camera 64 exceeds a preset threshold. If it is determined that the workpiece photography process is not complete, the rotary table 60 is rotated by a small angle (step S22), and the processes from step S14 to step S20 are repeated. On the other hand, if it is determined that the workpiece photography process is complete, the process proceeds to the next step S24.

[0078] As described above, the workpiece imaging process is performed, and the image acquisition unit 48 acquires multiple images of the workpiece W on the rotary table 60 from multiple viewpoints. The background image included in each image acquired at this time changes in accordance with the rotation angle of the rotary table 60. Note that the image acquisition unit 48 is an example of the image acquisition unit of the present invention.

[0079] After the workpiece imaging process is completed, the point cloud information generation unit 50 generates three-dimensional point cloud information of the workpiece W based on the multiple images acquired by the image acquisition unit 48 (step S24). In this embodiment, as described above, the background image included in each image changes in accordance with the rotation angle of the rotary table 60. Therefore, even when generating three-dimensional point cloud information of the workpiece W from multiple images using three-dimensional reconstruction technology such as NeRF or 3DGS, it is possible to obtain the three-dimensional point cloud information of the workpiece W with high accuracy without large errors. Note that the point cloud information generation unit 50 is an example of the point cloud information generation unit of the present invention.

[0080] Next, the position and orientation determination unit 52 reads the CAD data 72 of the workpiece W from the storage unit 36 ​​and generates a three-dimensional CAD model of the workpiece W based on the CAD data 72. Then, the position and orientation determination unit 52 determines the position and orientation of the workpiece W by fitting the three-dimensional CAD model of the workpiece W to the three-dimensional point cloud information of the workpiece W generated by the point cloud information generation unit 50 (step S26). Note that the position and orientation determination unit 52 is an example of the position and orientation determination unit of the present invention.

[0081] The position and orientation of the workpiece W determined by the position and orientation determination unit 52 are stored in the storage unit 36 ​​as workpiece position and orientation information, indicating the relative position and orientation of the workpiece W with respect to the three-dimensional coordinate measuring machine 10. As a result, the three-dimensional coordinate measuring machine 10 can recognize the relative positional relationship between the machine coordinate system defined in the three-dimensional coordinate measuring machine 10 and the workpiece coordinate system defined in the workpiece W, based on the workpiece position and orientation information stored in the storage unit 36, and the three-dimensional coordinate measuring machine 10 becomes capable of measuring each measurement element of the workpiece W. This concludes the flowchart.

[0082] In the flowchart shown in Figure 5, the virtual background model generation process is performed before the workpiece shooting process. However, this is not the only option; for example, the virtual background model generation process and the workpiece shooting process may be executed in real time in parallel (i.e., almost simultaneously).

[0083] [Positional Relationship of Each Part] In order to naturally display the background image generated by the background simulator unit 46 on the display 68 without a sense of discomfort, it is necessary that the positional relationships between the rotary table 60 (rotation center C1), the camera 64, and the display 68 in the three-dimensional coordinate measuring machine 10 and the corresponding parts of the background simulator unit 46 match. For this purpose, in the three-dimensional coordinate measuring machine 10, it is important to enable accurate acquisition of the relative positional relationship between the camera 64 and the rotary table 60 and the relative positional relationship between the camera 64 and the display 68 respectively in advance. This makes it possible to appropriately correct the positions of each part of the background simulator unit 46 based on these positional relationships. Hereinafter, a method for calculating the relative positional relationship between the camera 64 and the rotary table 60 and a method for calculating the relative positional relationship between the camera 64 and the display 68 will be described sequentially.

[0084] [Relative Positional Relationship Between Camera 64 and Rotary Table 60] A method for calculating the relative positional relationship between the camera 64 and the rotary table 60 in the three-dimensional coordinate measuring machine 10 will be described with reference to FIG. 6. FIG. 6 is a schematic diagram showing the relative positional relationship between the camera 64 and the rotary table 60 in the three-dimensional coordinate measuring machine 10.

[0085] As shown in FIG. 6, the world coordinate system Σ ｗ , the transformation matrix representing the position and orientation of the camera 64 is denoted as [R ｃ |t ｃ , and the transformation matrix representing the position and orientation of the rotary table 60 in the world coordinate system Σ ｗ is denoted as [R ｒｔ |t ｒｔ . In each transformation matrix, "R" represents a rotation matrix, "t" represents a translation vector, and the subscripts of each parameter (R, t) indicate corresponding elements. The world coordinate system Σ ｗ is a three-dimensional orthogonal coordinate system with an origin at any point in real space. The rotational coordinate system Σ ｒｔ takes any point O ｒｔ on the rotation axis of the rotary table 60 as its origin, the direction of the rotation axis of the rotary table 60 is defined as Z ｒｔ axis, and one of two mutually orthogonal directions perpendicular to the rotation axis of the rotary table 60 is defined as X ｒｔAxis, other direction is Y ｒｔ This is a three-dimensional Cartesian coordinate system. Camera coordinate system Σ ｃ This is the optical axis center O of camera 64. ｃ Let X be the origin, and the direction to the right of that origin. ｃ Axis, downward direction is Y ｃ The axis, the optical axis direction, is Z ｃ It is a three-dimensional Cartesian coordinate system with axes.

[0086] First, as a preliminary step to calculate the relative positional relationship between the camera 64 and the rotary table 60, the three-dimensional coordinate measuring machine 10 is set up to the state shown in Figure 6. Specifically, a jig 120 is placed on the rotary table 60. The jig 120 has at least three targets P ０ , P １ , P ２ These targets P ０ , P １ , P ２ These are connected to each other and integrated by a connecting member 122. ０ , P １ , P ２ These elements have a recognizable shape (spherical in this example) in the image captured by the camera 64 and are positioned apart from each other in a way that allows them to be identified.

[0087] Next, with the jig 120 placed on the rotary table 60, the rotary table 60 is rotated, and multiple images are captured by the camera 64 at multiple positions where the rotation angle of the rotary table 60 is different.

[0088] At this time, the rotating coordinate system Σ ｒｔ Target P ｉ The coordinates indicating the position (i = 0, 1, 2) are (x ｐｉ , y ｐｉ , z ｐｉ ) and target P on the captured image ｉ The coordinates (pixel coordinates) indicating the position of the (target image) are (u ｐｉ ,v ｐｉ If we assume that, then the following equation (1) holds true.

[0089] In equation (1), "K" is an internal parameter of the camera 64. Also, for simplicity, the distortion of the camera 64 has been omitted, but the target P on the shooting screen is shown. ｉ Pixel coordinates (u ｐｉ ,v ｐｉ For ), we can apply distortion correction beforehand before processing equation (1). Since distortion correction is well known, we will omit its explanation here.

[0090] Here, the world coordinate system Σ ｗ The origin of O ｗ and rotating coordinate system Σ ｒｔ The origin of O ｒｔ Treating them separately is redundant, therefore, the world coordinate system Σ ｗ The origin of O ｗ is a rotating coordinate system Σ ｒｔ The origin of O ｒｔ Assuming they are the same, the transformation matrix [R ｒｔ |t ｒｔ Translation vector "t" in ] ｒｔ " can be set to 0 (t ｒｔ = 0).

[0091] Also, the transformation matrix [R ｒｔ |t ｒｔ Rotation matrix R in ] ｒｔ is Z ｒｔ This is a rotation matrix that rotates by a rotation angle θ around an axis, and the rotation angle θ can be obtained from the value detected by the rotation angle detection unit 62 attached to the rotary table 60 and is known, so the rotation matrix R ｒｔ This is also a known matrix.

[0092] Therefore, equation (1) above can be expressed as equation (2) below.

[0093] In equation (2), "R ｒｔ (θ ｊ )" is the rotation matrix "R" in equation (1). ｒｔ This corresponds to the angle "θ" of the rotary table 60. ｊ The rotation matrix "R" corresponds to " ｒｔThis means that the values ​​of each element of " change. The subscript "j" corresponds to the number assigned to each captured image, and the rotation angle of the rotary table 60 when the j-th image is captured is "θ". ｊ This means that it is ".

[0094] Furthermore, in equation (2), the transformation matrix [R ｃ |t ｃ ] and each target P ｉ The coordinates (x) indicating the position ｐｉ , y ｐｉ , z ｐｉ ) is an unknown parameter, and all others are known parameters. Note that each target P ｉ The coordinates (x) indicating the position ｐｉ , y ｐｉ , z ｐｉ Although it is possible to determine this parameter accurately in advance, this model allows for simultaneous determination, so it is treated as an unknown parameter here.

[0095] Next, the error function E is defined as shown in equation (3). Note that in equation (3), the target P ｉ This is a generalized representation where the number of elements is N+1 and the number of images captured by camera 64 is M+1.

[0096] Then, by solving the error function E shown in equation (3) using a known nonlinear optimization method, the transformation matrix [R ｃ |t ｃ This allows us to determine the rotational coordinate system Σ. ｒｔ The position and orientation of camera 64 relative to this is "-R ｃ (Rotation matrix R ｃ (The inverse matrix of) and "-t ｃ (Translation vector t ｃ You can obtain the inverse vector of (the original vector).

[0097] In this way, the relative positional relationship between the camera 64 and the rotary table 60 in the three-dimensional coordinate measuring machine 10 can be calculated.

[0098] [Calculation of the relative positional relationship between camera 64 and display 68] The method for calculating the relative positional relationship between camera 64 and display 68 in the three-dimensional coordinate measuring machine 10 will be explained with reference to Figure 7. Figure 7 is a schematic diagram showing the relative positional relationship between camera 64 and display 68 in the three-dimensional coordinate measuring machine 10.

[0099] As shown in Figure 7, the coordinates (pixel coordinates) indicating the position of any point on the camera 64's screen (shooting screen) are (u ｃ ,v ｃ ) and the coordinates (pixel coordinates) indicating the position of any point on the screen (display screen) of the display 68 are (U ｄ , V ｄ )

[0100] To display images captured by a virtual camera 100 in a virtual space on a display 68 installed in the real space, and to ensure that images captured by a camera 64 installed in the real space are identical to images captured by the virtual camera 100 in the virtual space, it is necessary to determine the correspondence between the pixel coordinates on the screen of the camera 64 installed in the real space and the pixel coordinates on the screen of the display 68 installed in the real space.

[0101] Since the screens of the camera 64 and the display 68 are generally planar, we define them here using the homogeneous transformation matrix H as shown in equation (4).

[0102] One typical method for determining the homogeneous transformation matrix H is to follow the steps (1) to (3) below, but since this method is publicly known, a detailed explanation will be omitted here. (1) Display three or more markers at known pixel coordinates on the display 68. (2) Capture the display 68 with the camera 64 and calculate the pixel positions of the markers on the captured image. (3) Determine the homogeneous transformation matrix H by solving the following equation (5).

[0103] The homogeneous transformation matrix H obtained in this way represents the relative positional relationship between the camera 64 and the display 68.

[0104] It goes without saying that the homogeneous transformation matrix H may also be obtained by methods other than those described above.

[0105] [Displaying the background image on the display 68] By using the relative positional relationship between the camera 64 and the rotary table 60, and the relative positional relationship between the camera 64 and the display 68, obtained as described above, a transformation formula for projecting a three-dimensional point in the virtual space (a three-dimensional point defining the virtual background model 104) onto the screen of the display 68 can be obtained, as shown in equation (6) below. The coordinates indicating the position of the three-dimensional point in the virtual space are denoted as (x, y, z).

[0106] The background simulator unit 46 projects three-dimensional points in the virtual space (three-dimensional points defining the virtual background model 104) onto the screen of the display 68 based on the rotation angle of the rotary table 60, according to the conversion formula shown in equation (6). As a result, the display 68 can display a background image that changes in accordance with the rotation angle of the rotary table 60.

[0107] This makes it possible to make the background image included in the image captured by camera 64 in real space identical to the image captured by virtual camera 100 in virtual space, resulting in a natural-looking background image in the image captured by camera 64. As a result, it becomes possible to accurately determine the position and orientation of the workpiece W.

[0108] [Effects of the First Embodiment] In the first embodiment, in order to detect the position and orientation of the workpiece W placed on the rotary table 60, a camera 64 and a display 68 are positioned opposite each other on the rotary table 60, with the workpiece in between. The background image displayed on the display 68 is changed according to the rotation angle of the rotary table 60. Therefore, when the rotary table 60 is rotated and the workpiece W is photographed by the camera 64 at multiple positions where the rotation angle of the rotary table 60 is different, the background image included in the captured image by the camera 64 will change in accordance with the rotation angle of the rotary table 60.

[0109] Therefore, according to the first embodiment, even when generating three-dimensional point cloud information of the workpiece W from multiple captured images using three-dimensional reconstruction technology such as NeRF or 3DGS, it is possible to obtain the three-dimensional point cloud information of the workpiece W with high accuracy without increasing the error. As a result, the position and orientation of the workpiece W on the rotary table 60 can be determined accurately and easily without using a jig for setting the workpiece.

[0110] In the first embodiment, the function of the background simulator unit 46 was described as being implemented in software on the control unit 34 by the processor and memory of the computer 30, but as in the second embodiment described later, some or all of the functions of the background simulator unit 46 may be implemented in hardware.

[0111] <Second Embodiment> Next, a second embodiment of the present invention will be described. In the second embodiment, some of the functions of the background simulator unit 46 are implemented in hardware. The differences from the first embodiment will be described below, and the points that are common to the first embodiment will not be explained.

[0112] Figure 8 is a diagram illustrating a second embodiment. As shown in Figure 8, in the second embodiment, a background image is generated using a rotary table 106 and a real camera 110 placed in real space. Specifically, the real camera 110 is attached to the rotary table 106 installed in real space via a connecting member 108, so that the rotary table 106 and the real camera 110 can rotate together around a rotation center C3. The distance from the rotation center C1 of the rotary table 60 to the camera 64 is equal to the distance from the rotation center C3 of the rotary table 106 to the real camera 110. That is, the relative positional relationship between the rotary table 106 and the real camera 110 is equivalent to the relative positional relationship between the rotary table 60 and the camera 64. The real camera 110 is an example of another camera of the present invention. The rotary table 106 is an example of another rotary table of the present invention.

[0113] Then, the rotary table 106 is rotated together with the actual camera 110 around the rotation center C3, and the actual camera 110 takes pictures. At this time, the images captured by the actual camera 110 (hereinafter referred to as "actual images") may be a video taken continuously while the rotary table 106 is rotating, or they may be multiple still images taken discontinuously (periodically or non-periodically) each time the rotary table 106 rotates by a small angle. The actual images (360-degree omnidirectional images) captured by the actual camera 110 are stored in the storage unit 36 ​​as background data 70 associated with the rotation angle of the rotary table 106.

[0114] When workpiece photography is performed in the position and orientation detection process, the background simulator unit 46 acquires background data 70 from the storage unit 36. Based on the background data 70 and the rotation angle of the rotary table 60 detected by the rotation angle detection unit 62, the simulator unit 46 extracts an image from the background data 70 (actual images captured by the actual camera 110) that corresponds to the rotation angle of the rotary table 60 (an angle where the direction of rotation is opposite and the absolute value of the rotation angle is the same), and outputs the image converted as necessary to the display 68 as the background image.

[0115] In this explanation, we have described a case where, before the process of taking images with the camera 64 while rotating the rotary table 60 in the three-dimensional coordinate measuring machine 10 is performed (workpiece shooting process), a process of generating a background image using the rotary table 106 and the actual camera 110 installed in real space (background image generation process) is performed. However, the explanation is not limited to this, and the workpiece shooting process and the background image generation process may be performed simultaneously in real time.

[0116] Furthermore, in the second embodiment, it is necessary to adjust the relative positional relationship between the real camera 110 and the rotary table 106 in real space. Therefore, considering the ease of setup, the first embodiment described above is preferable.

[0117] <Third Embodiment> Next, a third embodiment will be described. In the first embodiment described above, the background image generated by the background simulator unit 46 is displayed on the display 68, and the camera 64 captures the workpiece W and the background image displayed on the display 68. In contrast, in the third embodiment, the background image that changes according to the rotation angle of the rotary table 60 is reflected in the captured image taken by the camera 64 by a method different from that of the first embodiment. The differences from the first embodiment will be described below, and the points that are common to the first embodiment will not be explained.

[0118] Figure 9 is a functional block diagram of the computer 30 in the third embodiment. Figure 10 is a diagram illustrating the position and orientation detection process in the third embodiment.

[0119] As shown in Figures 9 and 10, in the third embodiment, a background material 130 for chroma key compositing is placed on the three-dimensional coordinate measuring machine 10 at a position opposite the camera 64 across the rotary table 60, instead of the display 68 of the first embodiment (see Figure 10). This background material 130 is a single-color screen capable of chroma key compositing, such as a green screen. The position and orientation detection unit 44 is equipped with an image compositing unit 54, which will be described later. Note that the background material 130 is an example of a background material for chroma key compositing according to the present invention.

[0120] In the third embodiment, the workpiece W on the rotary table 60 is photographed by the camera 64 while the rotary table 60 is rotating. At this time, the background image in the image captured by the camera 64 includes the background material 130. The background image captured by the camera 64 is output to the image synthesis unit 54.

[0121] When the background simulator unit 46 obtains the rotation angle of the rotary table 60, it rotates the virtual camera 100 based on the rotation angle of the rotary table 60, similar to the first embodiment, and obtains an image captured by the virtual camera 100 of the virtual background model 104. The image captured by the virtual camera 100 is output to the image synthesis unit 54 as a background image.

[0122] The image synthesis unit 54 generates a composite image by combining an image obtained by removing the portion corresponding to the background material 130 from the captured image taken by the camera 64, using chroma key compositing, with a background image generated by the background simulator unit 46. The image synthesis unit 54 generates multiple composite images each time the rotary table 60 rotates and the camera 64 takes a picture of the workpiece W. The multiple composite images generated by the image synthesis unit 54 are output to the point cloud information generation unit 50. Note that the image synthesis unit 54 is an example of the image synthesis unit of the present invention.

[0123] In the third embodiment, since processing is performed by chroma key compositing as described above, it is not necessary to associate the pixel coordinates on the camera 64 screen with the pixel coordinates on the display 68 screen, as in the first embodiment. Therefore, the transformation formula for projecting a three-dimensional point in the virtual space (a three-dimensional point defining the virtual background model 104) onto the camera 64 screen is as shown in equation (7) below. In equation (7), the coordinates indicating the position of the three-dimensional point in the virtual space are (x, y, z), and the coordinates indicating the position of the point on the camera 64 screen are (u, v).

[0124] The point cloud information generation unit 50 generates three-dimensional point cloud information of the workpiece W based on a plurality of composite images generated by the image synthesis unit 54. The method for generating the three-dimensional point cloud information is the same as in the first embodiment, using three-dimensional reconstruction techniques such as NeRF or 3DGS. The processing after generating the three-dimensional point cloud information of the workpiece W is the same as in the first embodiment.

[0125] According to the third embodiment, a chroma key compositing process is used to generate multiple composite images by compositing a background image that changes according to the rotation angle of the rotary table 60 onto an image captured by the camera 64. This makes it possible to accurately obtain three-dimensional point cloud information of the workpiece W from the multiple composite images without using a jig for setting the workpiece. Therefore, the position and orientation of the workpiece W on the rotary table 60 can be determined accurately and easily.

[0126] <Other> In the workpiece imaging process in each of the embodiments described above, the rotation of the rotary table 60 may be configured to be performed automatically or manually. For example, when the rotation of the rotary table 60 is performed automatically, the control unit 34 may be equipped with a rotary table control unit (not shown), and the rotation of the rotary table 60 may be controlled by the rotary table control unit. Also, when the rotation of the rotary table 60 is performed manually, a manual operation unit may be provided for the user to manually rotate the rotary table.

[0127] Furthermore, in the workpiece imaging process, the camera 64's imaging operation may be configured to be performed automatically or manually. For example, when the camera 64's imaging operation is performed automatically, the control unit 34 may be equipped with a camera control unit (not shown), and the camera control unit may control the camera 64's imaging operation. In this case, the camera control unit may perform the camera 64's imaging operation in synchronization with the rotational movement of the rotary table 60, based on the rotation angle of the rotary table 60 detected by the rotation angle detection unit 62. Also, when the camera 64's imaging operation is performed manually, notification means (display means, sound output means, etc.) may be provided to inform the user of the camera 64's imaging timing.

[0128] Furthermore, in each of the embodiments described above, the images captured by the camera 64 are not limited to still images, but may also be moving images. When the camera 64 captures moving images, all or some of the image frames constituting the moving image may be selectively used as the captured images.

[0129] Although embodiments of the present invention have been described above, the present invention is not limited to the above examples, and various improvements and modifications may be made without departing from the spirit of the present invention.

[0130] 10... Three-dimensional coordinate measuring machine, 12... Probe head, 12a... Probe, 22... Gantry frame, 28... Drive controller, 30... Computer, 34... Control unit, 36... Memory unit, 40... Drive control unit, 42... Shape calculation unit, 44... Position and orientation detection unit, 46... Background simulator unit, 48... Image acquisition unit, 50... Point cloud information generation unit, 52... Position and orientation determination unit, 54... Image synthesis unit, 60... Rotating table, 62... Rotation angle detection unit, 64... Camera, 68... Display, 100... Virtual camera, 102... Virtual screen, 104... Virtual background model, 106... Rotating table, 108... Connecting member, 110... Actual camera, 120... Jig, 122... Connecting member, 130... Background material

Claims

1. A position and orientation detection device for detecting the position and orientation of a workpiece placed on a rotary table, comprising: a camera fixed at a position away from the rotary table; a display provided at a position opposite the camera with the workpiece in between; a background simulator unit that displays a background image on the display that changes in accordance with the rotation angle of the rotary table; an image acquisition unit that acquires a plurality of captured images taken by the camera at a plurality of positions with different rotation angles of the rotary table; a point cloud information generation unit that generates three-dimensional point cloud information of the workpiece based on the plurality of captured images; and a position and orientation determination unit that determines the position and orientation of the workpiece based on the three-dimensional point cloud information.

2. The position and orientation detection device according to claim 1, wherein the point cloud information generation unit generates three-dimensional point cloud information of the workpiece using NeRF or 3D Gaussian Platting.

3. The position and orientation detection device according to claim 1, wherein the background simulator unit rotates a virtual camera placed in the virtual space based on the rotation angle of the rotary table, and displays the background image generated based on the captured image taken by the virtual camera of the virtual background model in the virtual space on the display.

4. The position and orientation detection device according to claim 3, wherein the background simulator unit converts the captured image taken by the virtual camera based on the relative positional relationship between the camera and the display, and displays the converted captured image on the display as the background image.

5. The position and orientation detection device according to claim 1, wherein the background simulator unit displays the background image on the display based on the captured image taken by the other camera when the other rotating table is rotated with the other camera mounted on the other rotating table, which is positioned in real space.

6. A position and orientation detection device for detecting the position and orientation of a workpiece placed on a rotary table, comprising: a camera fixed at a position away from the rotary table; a background material for chroma key compositing provided at a position opposite the camera with the workpiece in between; an image acquisition unit for acquiring multiple captured images taken by the camera at multiple positions with different rotation angles of the rotary table; a background simulator unit for generating a background image that changes in accordance with the rotation angle of the rotary table; an image compositing unit for generating multiple composite images by compositing each of the multiple captured images with an image obtained by removing the portion corresponding to the background material from the captured image taken by the camera and the background image corresponding to the rotation angle of the rotary table; a point cloud information generation unit for generating three-dimensional point cloud information of the workpiece based on the multiple composite images; and a position and orientation determination unit for determining the position and orientation of the workpiece based on the three-dimensional point cloud information.

7. The position and orientation detection device according to any one of claims 1 to 6, wherein the rotary table is installed on the surface plate of a three-dimensional coordinate measuring machine, and the position and orientation determination unit determines the relative position and orientation of the workpiece with respect to the three-dimensional coordinate measuring machine.

8. A position and orientation detection method for detecting the position and orientation of a workpiece placed on a rotary table, comprising: a display step in which a camera and a display are positioned opposite each other with the workpiece in between, and a background image that changes in accordance with the rotation angle of the rotary table is displayed on the display; an image acquisition step in which a plurality of captured images are obtained by the camera at a plurality of positions where the rotation angles of the rotary table are different from each other; a point cloud information generation step in which three-dimensional point cloud information of the workpiece is generated based on the plurality of captured images; and a determination step in which the position and orientation of the workpiece are determined based on the three-dimensional point cloud information.

9. A position and orientation detection method for detecting the position and orientation of a workpiece placed on a rotary table, comprising: a background image generation step in which a camera and a background material for chroma key compositing are arranged opposite each other with the workpiece in between, and a background image generation step in which a background image that changes in accordance with the rotation angle of the rotary table is generated; an image acquisition step in which a plurality of captured images taken by the camera at a plurality of positions where the rotation angles of the rotary table are different from each other is acquired; an image compositing step in which, using chroma key compositing processing, a plurality of composite images are generated by compositing an image obtained by removing the portion corresponding to the background material from the captured image taken by the camera and the background image corresponding to the rotation angle of the rotary table with respect to each of the plurality of captured images; a point cloud information generation step in which three-dimensional point cloud information of the workpiece is generated based on the plurality of composite images; and a determination step in which the position and orientation of the workpiece are determined based on the three-dimensional point cloud information.

Images