Inspection equipment, inspection method, and program

Abstract

The object manufactured based on 3D CAD data is inspected by comparing it with the same 3D CAD data. [Solution] The inspection device 100 includes a control unit 110 that generates STL mesh data from 3D CAD data of an object, acquires scene point cloud data which is 3D data representing the shape of the object using a point cloud acquisition unit 150, matches each point in the master point cloud data generated from the STL mesh data with each point in the scene point cloud data, converts the coordinate system of the scene point cloud data to the coordinate system of the 3D CAD data, and performs measurements on measurement points set in the STL mesh data based on the scene point cloud data moved to the coordinate system of the 3D CAD data.

Description

Technical Field

[0001] The present invention relates to an inspection apparatus, an inspection method, and a program.

Background Art

[0002] In recent years, many products with complex shapes and parts have been designed based on 3D (Three-Dimensional) CAD (Computer Aided Design) data and manufactured, and methods for inspecting whether the products are manufactured according to the CAD data are known. For example, Patent Document 1 discloses a method of scanning an object to be inspected manufactured from CAD data with a three-dimensional scanner to obtain point cloud data, aligning the point cloud data with the CAD data, and inspecting the differences between the two data.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The method disclosed in Patent Document 1 basically aligns point cloud data with CAD data, and it was not clear how to perform inspection specifically after alignment. Therefore, in order to perform inspection using such a conventional technique, it was necessary to separately consider a specific inspection method.

[0005] The present invention has been made in view of the above circumstances, and an object thereof is to provide a specific means for inspecting an object manufactured based on 3D CAD data by comparing it with the 3D CAD data.

Means for Solving the Problems

[0006] )]]To achieve the above objective, the inspection apparatus according to the present invention is Generate STL mesh data from the 3D CAD data of the object, The point cloud acquisition unit acquires scene point cloud data, which is three-dimensional data showing the shape of the aforementioned object. Matching is performed between each point in the master point cloud data generated from the aforementioned STL mesh data and each point in the aforementioned scene point cloud data. The coordinate system of the aforementioned scene point cloud data is converted to the coordinate system of the aforementioned 3D CAD data. Measurements are performed on the measurement points set in the aforementioned STL mesh data, based on the scene point cloud data which has been moved to the coordinate system of the 3D CAD data. It is equipped with a control unit. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a specific means for inspecting an object manufactured based on 3D CAD data by comparing it with the said 3D CAD data. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing an example of the functional configuration of the inspection device according to this embodiment. [Figure 2] This is a flowchart of the registration process according to the embodiment. [Figure 3] This figure shows an example of how an image obtained by 3D imaging of an object is overlaid with STL mesh data based on 3D CAD data. [Figure 4] This figure shows an example of the original 3D CAD data. [Figure 5] This figure shows an example of 3D CAD data where only the part visible to the camera remains. [Figure 6] This figure shows an example of alignment points and measurement points set in the master point cloud data displayed in the 3D point cloud viewer. [Figure 7] This figure shows an example of measurement points when measuring the distance between two points. [Figure 8] This is a flowchart of the inspection process according to the embodiment. [Figure 9] This diagram illustrates a method for calculating displacement based on distance in the normal direction of the mesh. [Figure 10] This diagram illustrates a method for calculating displacement based on distance along the X or Y axis of the CAD coordinate system. [Figure 11] This figure shows an example of how displacements and defects in the object from the 3D CAD data are displayed. [Figure 12] This figure shows an example where the displacement verification area is set to an area extracted from the surface of the object's 3D CAD data. [Figure 13] This figure shows an example of measurement results when the displacement measurement area is set to an area extracted from the surface of the object's 3D CAD data. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

[0010] (Embodiment) The inspection apparatus 100 according to this embodiment includes, as shown in Figure 1, a control unit 110, a storage unit 120, a display unit 130, an operation input unit 140, and a point cloud acquisition unit 150. Three-dimensional CAD data of the object to be inspected (workpiece) is stored in the storage unit 120 in advance, and the control unit 110 creates STL (Standard Triangulated Language) mesh data from this three-dimensional CAD data. The control unit 110 then measures the three-dimensional shape of the object with the point cloud acquisition unit 150 to acquire point cloud data, and inspects the manufacturing accuracy of the object based on the STL mesh data and the point cloud data.

[0011] The control unit 110 is composed of a CPU (Central Processing Unit), an FPGA (Field Programmable Gate Array), etc. The control unit 110 executes registration processing, inspection processing, etc. described later according to the programs stored in the storage unit 120.

[0012] The storage unit 120 is composed of a RAM (Random Access Memory), a ROM (Read Only Memory), etc. In the storage unit 120, 3D CAD data of the object, programs for performing various processes in the control unit 110 (not only the programs for registration processing and inspection processing described later, but also programs for a 3D viewer that displays the three-dimensional shape of the object on the display unit 130 based on 3D CAD data and STL mesh data, and programs for a 3D point cloud viewer that displays the point cloud data acquired by the point cloud acquisition unit 150 on the display unit 130), etc. are stored in advance. Note that the 3D CAD data of the object may be downloaded to the storage unit 120 from a removable recording medium such as a USB (Universal Serial Bus) memory or a communication unit not shown.

[0013] The display unit 130 includes a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 130 displays the three-dimensional shape of the object based on the 3D CAD data, the image data acquired by the point cloud acquisition unit 150, the operation UI (User Interface) screen of the inspection device 100, etc. However, the inspection device 100 may include these displays as the display unit 130, or may include the display unit 130 as an interface for connecting an external display. When the inspection device 100 includes the display unit 130 as an interface, the three-dimensional shape, image data, etc. of the object are displayed on an external display connected via the display unit 130.

[0014] The operation input unit 140 is a device that receives a user's operation input to the inspection device 100, and examples thereof include a keyboard, a mouse, a touch panel, and the like. The inspection device 100 receives instructions from the user via the operation input unit 140.

[0015] The point cloud acquisition unit 150 includes a three-dimensional camera or the like capable of three-dimensional (3D) imaging of an object, and acquires point cloud data, which is three-dimensional data indicating the shape of the photographed object. The point cloud data obtained by the three-dimensional camera is a set of three-dimensional coordinate (XYZ coordinate) data obtained by discretizing a three-dimensional shape in terms of pixels. Hereinafter, in order to distinguish it from the point cloud data obtained from the STL mesh data described later, the point cloud data obtained by 3D imaging is referred to as scene point cloud data, and the point cloud data generated from the STL mesh data (for example, by extracting the vertices of the triangles of each mesh) is referred to as master point cloud data. In addition, the point cloud acquisition unit 150 can not only acquire point cloud data, but also acquire two-dimensional image data of the imaging target, similar to a normal camera.

[0016] Examples of the device configuration of such a point cloud acquisition unit 150 include those composed of a stereo camera with two cameras, those composed of one camera and one projector, those composed of two cameras and one projector, etc., but the device configuration of the point cloud acquisition unit 150 may be any configuration.

[0017] The functional configuration of the inspection device 100 has been described above. Next, the registration process executed by the control unit 110 of the inspection device 100 will be described with reference to FIG. 2. This process is a process for registering the master point cloud data of the object (created from the CAD data of the object), the scene point cloud data (created by photographing the object with a 3D camera), measurement points, etc. when inspecting the manufacturing accuracy of the object. When the user instructs the inspection device 100 to start the registration process, the execution of the registration process is started.

[0018] First, the control unit 110 creates STL mesh data from the 3D CAD data of the object to be inspected (step S101). At this time, the mesh size is set according to the accuracy that is actually required for inspection (for example, the length of the longest side is set to 1 mm). It is desirable to set this mesh size so that the granularity of the scene point cloud data obtained based on the resolution of the point cloud acquisition unit 150 and the granularity of the STL mesh data are approximately the same. If the mesh granularity of the two differs significantly, it is possible to improve accuracy and optimize the matching time by performing a voxelization process to match the granularity. Then, all the vertices of each triangle that make up the STL mesh data are extracted to create initial master point cloud data. The STL mesh data created in step S101 is also called initial master STL data.

[0019] Then, the control unit 110 uses the point cloud acquisition unit 150 to perform 3D imaging of the object in a posture similar to that of an object during actual inspection (i.e., an object on a production line) and acquires scene point cloud data (step S102). As a result, scene point cloud data consisting of point cloud data of the portion of the object visible to the point cloud acquisition unit 150 is acquired.

[0020] The control unit 110 then overlays the acquired scene point cloud data and STL mesh data and calculates a 4x4 RT (Rotate Transform) matrix to convert (rotate and translate) the coordinate values ​​of the master point cloud data to the coordinate values ​​of the scene point cloud data (step S103). However, there are two ways to perform the process in step S103: automatically and manually, and either method may be used. The automatic method involves the control unit 110 extracting initial master point cloud data from the STL mesh data (for example, by extracting the vertices of each triangle) and matching it with the scene point cloud data (for example, rough alignment using normal feature points RANSAC (Random Sample Consensus) followed by detailed alignment using ICP (Iterative Closest Point)). In this case, the control unit 110 calculates the RT matrix to convert the coordinate values ​​of the master point cloud data to the coordinate values ​​of the scene point cloud data using ICP. The manual execution method, as shown in Figure 3, involves using a 3D viewer to rotate, move, enlarge, and reduce the STL mesh data 302 (master point cloud data) based on the object's 3D CAD data, by overlaying it onto the object's 3D captured image 301 (scene point cloud data) using the operation input unit 140 (mouse, etc.). With this method, accurate shape matching is difficult unless the camera parameters, such as the focal length of the virtual camera used to display the CAD data shown in computer graphics, are matched to the parameters used in the actual camera. However, if these camera parameters cannot be adjusted, the user will determine an approximate camera position and obtain the FOV (Field of View) and line of sight direction from the origin of that camera coordinate system. In this case, the control unit 110 calculates the RT matrix from the CAD coordinates to the camera coordinate system, which is the coordinate system of the scene point cloud, from the displacement amount of the master point cloud data that has been manually matched to the scene point cloud data in this way.

[0021] Then, the control unit 110 deletes the STL mesh that is not visible from the camera viewpoint of the point cloud acquisition unit 150 from the initial master STL data to create edited master STL data, and extracts the vertices of each triangle contained in the edited master STL data to create master point cloud data (called edited master point cloud data when distinguishing it from the initial master point cloud data) to be used in the inspection process described later (step S104). As a result, for example, in the original 3D CAD data 320 (initial master STL data) as shown in Figure 4, if the camera of the point cloud acquisition unit 150 was pointed from the upper left back to the lower right front of the object, the 3D CAD data 321 (edited master STL data) that remains with only the part visible from the camera will take the form shown in Figure 5. The deletion of this invisible STL mesh data is automatically performed by the control unit 110 using the camera origin coordinates and the FOV (Field of View) and line of sight direction from the camera coordinate origin.

[0022] Furthermore, if it is unnecessary to erase such invisible STL meshes (for example, when comparing the entire STL mesh data obtained from CAD data (initial master point cloud data) with the entire scene point cloud data obtained from 3D imaging for inspection), the processing in steps S103 to S104 is unnecessary. In this case, all vertices of the triangles in the STL mesh data (initial master point cloud data) can be used as the master point cloud data for the inspection process. However, in this case, it is important to note that the degree of matching between the master point cloud data and the scene point cloud data will be low because the master point cloud data includes all surface data of the object visible from all 360 degrees, while the scene point cloud data only includes surface data visible from the point cloud acquisition unit 150. Also, if the object is thin, the master point cloud data includes data for the back surface which has almost the same shape as the surface data, which can lead to the problem of not knowing which side of the master point cloud data the scene point cloud data matches.

[0023] Next, the control unit 110 sets the alignment points between the scene point cloud data and the master point cloud data (step S105). Specifically, the control unit 110 displays the master point cloud data 303 of the object in a 3D point cloud viewer as shown in Figure 6, and the user specifies a point near the center of the area to be used for alignment using the operation input unit 140 (for example, by clicking with the mouse). Then, the control unit 110 sets a rectangular range 304 of any size in the XYZ directions centered on the point specified by the user, and uses the master point cloud data 303 included in this range 304 as the alignment point with the scene point cloud data. The positions used as alignment points are (1) positions that serve as a reference for measurement, (2) positions where deformation is expected to be minimal and the shape is expected to be stable, etc., and the user specifies these positions using the operation input unit 140. In addition, the alignment points need to have shape features that facilitate 3D shape matching (for example, a shape that is as asymmetrical as possible).

[0024] The control unit 110 then sets the measurement points and measurement items based on the information entered in the operation input unit 140 (step S106). Specifically, in setting the measurement points, the control unit 110 displays the master point cloud data 303 of the object on the display unit 130 using a 3D point cloud viewer. The user then specifies a certain range of desired locations from the displayed master point cloud data 303 using the operation input unit 140, and this range (the area contained within a three-dimensional object (cuboid, cylinder, sphere, etc.) specified by clicking or dragging with the mouse, for example) is set as the measurement range. In other words, the measurement points are set to be the range (measurement range) contained within a three-dimensional object (cuboid, cylinder, sphere, etc.) that includes at least one point in the master point cloud data 303 specified by the user. The control unit 110 then sets measurement items for each measurement point (measurement range). Measurement items include (1) displacement amount (difference between master point cloud data and scene point cloud data (in the normal direction of the STL mesh, or in the XYZ axis direction)), (2) measured value (distance between two set measurement planes, diameter of holes present in the set measurement plane, etc.), and (3) burrs and chips (check for burrs and chips present in the set measurement range). The measurement planes referred to here are derived from the master point cloud data included in the user-specified measurement range. When the measurement item is the distance between two points (between measurement range A and measurement range B), the plane closest to each point in the scene point cloud data matched to the master point cloud data included in measurement range A is determined using the least squares method, and this is designated as measurement plane A. Similarly, the plane closest to each point in the scene point cloud data obtained from measurement range B is determined using the least squares method, and this is designated as measurement plane B. These measurement planes (measurement plane A and measurement plane B) are also called scene measurement planes because they are measurement planes derived from scene point cloud data.

[0025] Please note that the method for setting measurement points (measurement range) differs for each measurement item, so further explanation is provided below. When the measurement item is displacement, the user sets the area to be measured as the measurement point. For example, if the user wants to check the amount of deformation from the CAD data of the entire object, as shown in Figure 6, the user sets the displacement check area 305 as the measurement point so that it covers the entire object. Furthermore, if the measurement item is the distance between two measurement points, the user sets those two measurement points. For example, as shown in Figure 7, the user sets measurement area 307 as the first measurement area and measurement area 308 as the second measurement area. Furthermore, if the measurement item is burrs or chips, the user sets a confirmation area to detect burrs or chips as a measurement point. For example, if the user wants to check for burrs or chips at the end of a specific feather of an object, the user sets the burr / chip confirmation area 306 as a measurement point, as shown in Figure 6. Furthermore, if the measurement item is hole diameter, the user sets the measurement range to include the area containing the hole whose diameter they want to measure.

[0026] Returning to Figure 2, the control unit 110 then saves the edited master STL data and RT matrix to the storage unit 120 (step S107), and the registration process ends. With this, the master data for measurement (master point cloud data, data for aligning the master and the scene, measurement points, measurement items, etc.) is registered, and preparations for the inspection process described later are complete.

[0027] Next, the inspection process, which uses the master data registered in the registration process (master point cloud data, RT matrix for coarse alignment of the master and scene, measurement points, measurement items, etc.) to inspect the object, will be explained with reference to Figure 8. When the user instructs the inspection device 100 to start the inspection process, the execution of the inspection process begins. This instruction does not have to be given by a human; for example, it can be given by a higher-level PLC (Programmable Logic Controller) or a robot controller. Since the inspection process can perform a 100% inspection of the manufactured objects, the user can, for example, install the inspection device on a factory production line so that all manufactured objects can be photographed by the point cloud acquisition unit 150, and then instruct the start of the inspection process.

[0028] First, the control unit 110 uses the point cloud acquisition unit 150 to 3D photograph the object (on the production line) and acquire point cloud data (scene point cloud data) (step S201). Then, scene point cloud data of the object is acquired at the resolution of the 3D camera equipped in the point cloud acquisition unit 150.

[0029] The control unit 110 then performs voxelization to match the granularity of the STL mesh data (master point cloud data) and the scene point cloud data (step S202). For example, if the mesh size of the STL mesh data can only be fined down to 1 mm, but the point cloud can be acquired in units of 0.2 mm as dots when acquiring the scene point cloud data, then voxelization of the scene point cloud data using a grid with 1 mm intervals is performed to match the granularity of the scene point cloud data to that of the STL mesh data. Conversely, if the mesh size of the STL mesh data is finer than the dots used when acquiring the scene point cloud data, the STL mesh data may be voxelized to match the granularity. In addition, to speed up alignment, both the STL mesh data (master point cloud data) and the scene point cloud data may be voxelized to an appropriate size using grids with the same intervals as needed. The granularity of this voxelization also affects the matching processing time between the master point cloud and the scene point cloud, so that time is also a consideration. For example, increasing the granularity can shorten the matching processing time.

[0030] Next, the control unit 110 performs alignment so that the scene point cloud data is superimposed on the master point cloud data (S203). At this time, the RT matrix calculated in step S103 of the registration process (Figure 2) is used. Specifically, by multiplying the scene point cloud data by the inverse of the RT matrix, the scene point cloud data can be moved to the vicinity of the master point cloud data. Using this position as the initial value, detailed alignment is performed by ICP processing between the master point cloud data for alignment registered in the master data and the point cloud data obtained by moving the scene point cloud data to the vicinity of the master point cloud data using the inverse of the RT matrix. Alternatively, instead of using master point cloud data for alignment, it is possible to perform coarse alignment between the master point cloud data and the scene point cloud data using appropriate methods (e.g., RANSAC), followed by detailed alignment using ICP. Coarse alignment using RANSAC involves randomly extracting a predetermined number of meshes (e.g., several dozen) from the STL mesh data, and using the normal vectors of each of these extracted meshes as features to match them with the scene point cloud data. This process is repeated by changing the number of meshes randomly extracted from the STL mesh data several times, and when the best match is obtained, the control unit 110 determines that the points (points in the master point cloud) corresponding to the predetermined number of meshes extracted at the time the best match was obtained correspond to each point in the scene point cloud that was matched with each of those points.

[0031] By performing detailed alignment (for example, detailed alignment using ICP), the control unit 110 calculates a coordinate transformation matrix M in step S203 that converts the coordinates of each point in the scene point cloud data to the coordinates of each point in the master point cloud data. In other words, the coordinate transformation matrix M is a matrix that converts the coordinate system of the scene point cloud data to the coordinate system of the 3D CAD data. The control unit 110 can perform matching processing relatively quickly by photographing the workpieces (objects) flowing one after another on the factory line and performing ICP using the RT matrix obtained in S103 as an initial value, thereby enabling the calculation of the M matrix at high speed. In order to further shorten this matching time, voxelization to an appropriate granularity may be performed in step S202.

[0032] Then, the control unit 110 transforms the coordinates of each point in the scene point cloud data before voxelization using the coordinate transformation matrix M, thereby converting them to coordinate values ​​in the coordinate system of the master point cloud data (step S204). In other words, the control unit 110 can move the coordinate values ​​of each point in the scene point cloud data to the vicinity of the coordinate values ​​of the STL mesh data by transforming the coordinate system of the scene point cloud data using the coordinate transformation matrix M.

[0033] Then, the control unit 110 performs measurements based on the settings configured in the registration process (step S205). As mentioned above, the measurement items include displacement, burrs / chips, distance between two points, hole diameter, etc., but in all cases, the control unit 110 performs measurements based on scene point cloud data (moved to the coordinate system of the 3D CAD data) that corresponds to the measurement points (measurement range) set in the master point cloud data. Here, we will first explain the case where the measurement item is displacement. In this case, the control unit 110 will determine the distance to the scene point cloud data (this is the displacement amount calculated as the measurement value) for each mesh in the edited master STL data included within the verification area 305 set by the user as a measurement point. For example, when determining the displacement amount for mesh 361 in the edited master STL data shown in Figure 9, the control unit 110 searches for the scene point cloud 311 in the normal direction of mesh 361, and calculates the displacement amount (displacement amount from the design position to the manufacturing position) as the average value 370 of the distance 312 to the searched scene point cloud 311. Here, the scene point cloud 311 is the scene point cloud 311 after being transformed into coordinates in the coordinate system of the master point cloud data by the coordinate transformation matrix M in step S204. In short, the control unit 110 uses a point in the edited master STL data as a reference point and calculates the displacement amount as the distance to the point in the scene point cloud data that matches this reference point. In other words, for each mesh corresponding to the master point cloud data matched to the scene point cloud data, the distance from each scene point cloud to the mesh surface in the perpendicular direction is calculated as a measured value (displacement) for the scene point clouds included in the projection in the normal direction of the mesh.

[0034] The area used to search for scene point cloud 311 is defined as the scene point cloud 311 contained within a triangular prism (a triangular prism with mesh 361 as its base and its height in the direction normal to mesh 361) obtained by projecting mesh 361 in the direction normal to mesh 361. This search area does not necessarily have to be the area onto which mesh 361 itself is projected; it may be enlarged, reduced, or deformed (for example, into an area onto which a circle centered on the centroid of mesh 361 is projected). In addition, the search is performed in both positive and negative directions in the height direction, but if no scene point cloud 311 is found after searching beyond a predetermined threshold distance, it is assumed that there is no scene point cloud 311 corresponding to that mesh 361.

[0035] Furthermore, the calculated displacement is usually the average value of the distances in the normal direction of each mesh in the edited master STL data, which is 370. However, it is not necessary to limit the calculation to the normal direction, and the distance can be calculated in any direction. In cases where it is desired to obtain displacement in directions other than the normal direction, such as the XYZ axis direction of the CAD data, as shown in Figure 10, for example, if the displacement in the X-axis direction is desired, the average value of the distances 312 from mesh 361 to scene point cloud 311 in the X-axis direction, which is 371, is calculated as the displacement. Similarly, if the displacement in the Y-axis direction is desired, the average value of the distances 372 from mesh 361 to scene point cloud 311 in the Y-axis direction is calculated as the displacement. In other words, for each mesh corresponding to the master point cloud data matched to the scene point cloud data, the distance from each scene point cloud to the mesh surface in the direction of the selected axis (X-axis, Y-axis, or Z-axis) is calculated as the measured value (displacement) for the scene point clouds included in the projection of the 3D CAD data in the direction of either the X-axis, Y-axis, or Z-axis. In this case as well, the search area does not necessarily have to be the area obtained by projecting the mesh 361 onto each axis; it can be enlarged, reduced, or deformed. Furthermore, the search is performed in both positive and negative directions in the height direction. However, if no scene point cloud 311 is found after searching beyond a predetermined threshold, it is assumed that no scene point cloud 311 exists corresponding to that mesh 361.

[0036] The control unit 110 then displays the scene point cloud 311, colored according to the calculated displacement amount, on the display unit 130. The color assigned to each point in the scene point cloud 311 is arbitrary, but in this embodiment, the displacement amount at each point is represented by coloring them in accordance with the color wheel. Specifically, the scene point cloud 311 with a displacement of 0 is colored green on the color wheel, the color of the scene point cloud 311 changes from yellow to red as the displacement amount is positive and large, and the color of the scene point cloud 311 changes from light blue to blue as the displacement amount is negative and its absolute value is large.

[0037] As a result, as shown in Figure 11, the amount of displacement between the design data and the manufactured product can be grasped at a glance. Note that Figure 11 is a grayscale image, so the difference in color is difficult to see, but in reality, the parts that are "indented" (areas with negative displacement) are displayed in blue, and the parts that are "bulging" (areas with positive displacement) are displayed in red. The control unit 110 displays this color map, so it is immediately clear that the areas with colors close to green have a small amount of displacement, and the areas with red or blue have a large amount of displacement. The above explains the case when the measurement item is the amount of displacement.

[0038] Next, we will explain the case where the measurement item is a burr or chip. In this case as well, the same processing as when determining the displacement amount is performed, but if the scene point cloud 311 is not found (does not exist) even after searching both positive and negative directions within a predetermined threshold range for each mesh (for example, mesh 361 shown in Figure 9) in the verification area where the user has specified a burr or chip as a measurement point in the edited master STL data, then the scene point cloud 311 corresponding to that mesh 361 is considered not to exist, and the point cloud is not displayed as shown as a "chip" in Figure 11. As a result, even in locations where CAD data should exist in the camera's field of view, the point cloud will not be displayed at that location, and the user will be able to understand that some kind of "chip" has occurred there. In addition to simply not displaying the point cloud, it is also possible to clearly indicate that there is a "chip". That is, when the control unit 110 outputs a "chip", it may choose not to display the point cloud at that location, or it may choose to indicate that a "chip" exists. Conversely, among the scene point cloud 311 in the verification area designated by the user as a measurement point for detecting burrs and defects, points that were not found (did not match any points in the master point cloud data) even after searching both positive and negative directions for the normal direction of each mesh in the edited master STL data present in the verification area (for example, mesh 361 shown in Figure 9) up to a predetermined threshold range are colored gray with a saturation of 0. These gray-colored areas do not exist in the CAD data but do exist in the object, so they can be considered either noise from the image capture or, in the case of a casting workpiece, if they are of a certain size, they can be considered "burrs". As a result, even in areas that should not exist in the CAD data, the scene point cloud will be displayed at that location, allowing the user to understand that "burrs" have occurred there. The above explains the case where the measurement item is burrs or chips.

[0039] Next, we will explain the case where the measurement item is the distance between two points. In this case, the control unit 110 will determine the distance between two measurement points set by the user. In step S106 of the registration process (Figure 2), the user sets two measurement points (a measurement area 307 indicating the first measurement point and a measurement area 308 indicating the second measurement point) on the master point cloud data, for example, as shown in Figure 7. Then, in step S205 of the inspection process (Figure 8), the control unit 110 calculates the distance between the measurement plane (first measurement plane) obtained by least squares from the scene point cloud data included in the first measurement area where the measurement points are set (a measurement plane), and the measurement plane (second measurement plane) obtained by least squares from the scene point cloud data included in the second measurement area (a measurement plane), from the 3D captured scene point cloud data. In other words, the control unit 110 acquires two measurement points (a first measurement area and a second measurement area) from the master point cloud data, and calculates the distance between the two measurement planes, which is determined by the least squares method from the scene point cloud data contained in each (obtained by searching from each measurement area), as a measured value. When determining the distance between two measurement points in this way, the control unit 110 calculates the distance between the two measurement planes as a measured value 309, based on the normal direction of either of the two measurement planes (the first measurement plane corresponding to the first measurement area 307 and the second measurement plane corresponding to the second measurement area 308) which are calculated by the least squares method from the scene point clouds contained in each of the two measurement points (the first measurement area 307 and the second measurement area 308) acquired in the registration process. Furthermore, the direction in which the distance between two points is calculated is not limited to the normal direction of either of the two measurement planes determined as described above. Alternatively, the normal direction of the STL mesh contained in each of the two measurement regions in the edited master STL data may be set to the same direction, and the distance from the first measurement plane to the second measurement plane along the normal direction of the STL mesh may be calculated as the measured value, or the distance to any of the XYZ axes may be calculated. In some cases, stable measurements can be performed by determining the normal direction based on CAD data.

[0040] In summary, we have explained three types of measurement items (displacement, burrs / chips, and distance between two points), but other measurements (such as circle diameter) can be performed using a similar approach. Returning to Figure 8, after the measurement in step S205, the control unit 110 determines whether or not there has been an instruction to terminate (step S206). For example, if the user gives an instruction to terminate using the operation input unit 140 when terminating an inspection on the line, the determination in step S206 will be Yes. If there is an instruction to terminate (step S206; Yes), the control unit 110 terminates the inspection process. If there is no instruction to terminate (step S206; No), the control unit 110 returns to step S201 and continues the inspection process.

[0041] Through the inspection process described above, the control unit 110 can inspect an object manufactured based on CAD data by comparing it with the CAD data. Specifically, the control unit 110 can calculate the amount of displacement from the CAD data, display a displacement map (color map) by coloring the scene point cloud according to the amount of displacement, detect areas where the master point cloud data and the scene point cloud data could not be matched as burrs or defects, and measure the distance between two points.

[0042] By measuring in this way, it becomes easy to measure the distance between two points on an object that would be difficult to measure directly. Furthermore, it is possible to accurately determine how the values ​​in the 3D CAD data (design values) actually correspond to the values ​​in the manufactured object (how accurately it was manufactured). Furthermore, the inspection time includes the time for measuring the point cloud of the object, the time for matching the master point cloud data with the scene point cloud data, and the time for outputting the measured values ​​at the measurement points. For typical industrial products, this measurement process can be completed in 3 to 5 seconds.

[0043] In the above explanation, when measuring displacement, a verification area 305 covering the entire object was set as shown in Figure 6. However, as shown in Figure 12, an area cut out from any face included in the 3D CAD data of the object may be set as the verification area 315. In this case, as shown in Figure 13, the displacement amount is shown in a color map within the area 316 corresponding to the set verification area 315.

[0044] Furthermore, each function of the control unit 110 of the inspection device 100 can also be performed by a computer such as a regular PC (Personal Computer). Specifically, in the above embodiment, it was described that the program for the processing performed by the control unit 110 of the inspection device 100 is pre-stored in the ROM of the storage unit 120. However, a computer capable of realizing the above functions may be configured by distributing the program on a computer-readable recording medium such as a flexible disk, CD-ROM (Compact Disc Read Only Memory), DVD (Digital Versatile Disc), MO (Magneto-Optical Disc), memory card, or USB memory, and then loading and installing the program into the computer.

[0045] While preferred embodiments of the present invention have been described above, various embodiments and modifications are possible without departing from the broad spirit and scope of the present invention. Furthermore, the embodiments described above are for illustrative purposes only and do not limit the scope of the present invention. In other words, the scope of the present invention is indicated not by the embodiments, but by the claims. Various modifications made within the scope of the claims and the equivalent scope of the meaning of the invention are considered to be within the scope of this invention.

[0046] (Note) The invention described in the original claims of this application is listed below.

[0047] (Note 1) Generate STL mesh data from the 3D CAD data of the object, The point cloud acquisition unit acquires scene point cloud data, which is three-dimensional data showing the shape of the aforementioned object. Matching is performed between each point in the master point cloud data generated from the aforementioned STL mesh data and each point in the aforementioned scene point cloud data. The coordinate system of the aforementioned scene point cloud data is converted to the coordinate system of the aforementioned 3D CAD data. Measurements are performed on the measurement points set in the aforementioned STL mesh data, based on the scene point cloud data which has been moved to the coordinate system of the 3D CAD data. Equipped with a control unit, Inspection device.

[0048] (Note 2) The control unit, Edited master STL data is created using only the portion of the STL mesh data that can be captured by the point cloud acquisition unit, and the edited master STL data is used as the STL mesh data. The inspection device described in Appendix 1.

[0049] (Note 3) The control unit, The measurement points are defined as a range contained within a rectangular prism, cylinder, or sphere that includes at least one point in the master point cloud data specified by the user. The inspection device described in Appendix 1.

[0050] (Note 4) The control unit, For each mesh corresponding to the master point cloud data matched to the scene point cloud data, the distance from each scene point cloud to the mesh surface in the perpendicular direction is calculated as a measured value for the scene point clouds included in the projection in the normal direction of the mesh. The inspection device described in Appendix 2.

[0051] (Note 5) The control unit, For each mesh corresponding to the master point cloud data matched to the scene point cloud data, the distance from each scene point cloud to the mesh surface in the direction of the X, Y, or Z axis of the 3D CAD data is calculated as a measured value. The inspection device described in Appendix 2.

[0052] (Note 6) The control unit, A first measurement region and a second measurement region are obtained from the master point cloud data. The distance between the first measurement plane and the second measurement plane obtained from the scene point cloud data included in the second measurement region is calculated as a measured value in the normal direction obtained from the first measurement plane obtained from the scene point cloud data included in the first measurement region, or in the normal direction obtained from the STL mesh in the edited master STL data included in the first measurement region. The inspection device described in Appendix 2.

[0053] (Note 7) The control unit, The first measurement area and the second measurement area are obtained from the aforementioned edited master STL data. The distance between a first measurement plane obtained from the scene point cloud data included in the first measurement area and a second measurement plane obtained from the scene point cloud data included in the second measurement area is calculated as a measured value along the X, Y, or Z axis of the 3D CAD data. The inspection device described in Appendix 2.

[0054] (Note 8) The control unit, When calculating the aforementioned measured value, Of the edited master STL data, meshes where the scene point cloud data does not exist within a certain distance are output as "missing points". The inspection device described in Appendix 4 or 5.

[0055] (Note 9) The control unit, When calculating the aforementioned measured value, Of the aforementioned scene point cloud data, the scene point cloud data for which the edited master STL data does not exist within a certain distance is output as "variables". The inspection device described in Appendix 4 or 5.

[0056] (Note 10) The control unit, Generate STL mesh data from the 3D CAD data of the object, The point cloud acquisition unit acquires scene point cloud data, which is three-dimensional data showing the shape of the aforementioned object. Matching is performed between each point in the master point cloud data generated from the aforementioned STL mesh data and each point in the aforementioned scene point cloud data. The coordinate system of the aforementioned scene point cloud data is converted to the coordinate system of the aforementioned 3D CAD data. Measurements are performed on the measurement points set in the aforementioned STL mesh data, based on the scene point cloud data which has been moved to the coordinate system of the 3D CAD data. Testing method.

[0057] (Note 11) In the control unit, Generate STL mesh data from the 3D CAD data of the object, The point cloud acquisition unit acquires scene point cloud data, which is three-dimensional data showing the shape of the aforementioned object. Matching is performed between each point in the master point cloud data generated from the aforementioned STL mesh data and each point in the aforementioned scene point cloud data. The coordinate system of the aforementioned scene point cloud data is converted to the coordinate system of the aforementioned 3D CAD data. Measurements are performed on the measurement points set in the aforementioned STL mesh data, based on the scene point cloud data which has been moved to the coordinate system of the 3D CAD data. A program that performs a process. [Explanation of Symbols]

[0058] 100... Inspection device 110... Control Unit 120...Storage section 130...Display section 140... Operation input section 150...Point cloud acquisition section 301...3D captured image 302…STL Mesh Data 303...Master point cloud data 304... range 305,306,315…Confirmation area 307,308… Measurement range 309... Measured value 311... Scene point cloud 316…area 320, 321… 3D CAD data 361... Mesh 370,371,372…distance

Claims

1. STL mesh data is generated from the 3D CAD data of the object. The point cloud acquisition unit acquires scene point cloud data, which is three-dimensional data showing the shape of the aforementioned object. Matching is performed between each point in the master point cloud data generated from the aforementioned STL mesh data and each point in the aforementioned scene point cloud data. The coordinate system of the aforementioned scene point cloud data is converted to the coordinate system of the aforementioned 3D CAD data. Measurements are performed on the measurement points set in the STL mesh data based on the scene point cloud data which has been moved to the coordinate system of the 3D CAD data. Equipped with a control unit, Inspection device.

2. The control unit, Edited master STL data is created using only the portion of the STL mesh data that can be captured by the point cloud acquisition unit, and the edited master STL data is used as the STL mesh data. The inspection apparatus according to claim 1.

3. The control unit, The measurement points are defined as a range contained within a rectangular prism, cylinder, or sphere that includes at least one point in the master point cloud data specified by the user. The inspection apparatus according to claim 1.

4. The control unit, For each mesh corresponding to the master point cloud data matched to the scene point cloud data, the distance from each scene point cloud to the mesh surface in the perpendicular direction is calculated as a measured value for the scene point clouds included in the projection in the normal direction of the mesh. The inspection apparatus according to claim 2.

5. The control unit, For each mesh corresponding to the master point cloud data matched to the scene point cloud data, the distance from each scene point cloud to the mesh surface in the direction of the X, Y, or Z axis of the 3D CAD data is calculated as a measured value. The inspection apparatus according to claim 2.

6. The control unit, A first measurement region and a second measurement region are obtained from the master point cloud data. The distance between the first measurement plane and the second measurement plane obtained from the scene point cloud data included in the second measurement region is calculated as a measured value in the normal direction obtained from the first measurement plane obtained from the scene point cloud data included in the first measurement region, or in the normal direction obtained from the STL mesh in the edited master STL data included in the first measurement region. The inspection apparatus according to claim 2.

7. The control unit, The first measurement area and the second measurement area are obtained from the edited master STL data. The distance between a first measurement plane obtained from the scene point cloud data included in the first measurement area and a second measurement plane obtained from the scene point cloud data included in the second measurement area is calculated as a measured value along the X, Y, or Z axis of the 3D CAD data. The inspection apparatus according to claim 2.

8. The control unit, When calculating the aforementioned measured value, Of the edited master STL data, meshes where the scene point cloud data does not exist within a certain distance are output as "missing points". The inspection apparatus according to claim 4 or 5.

9. The control unit, When calculating the aforementioned measured value, Of the aforementioned scene point cloud data, the scene point cloud data for which the edited master STL data does not exist within a certain distance is output as "variables". The inspection apparatus according to claim 4 or 5.

10. The control unit, STL mesh data is generated from the 3D CAD data of the object. The point cloud acquisition unit acquires scene point cloud data, which is three-dimensional data showing the shape of the aforementioned object. Matching is performed between each point in the master point cloud data generated from the aforementioned STL mesh data and each point in the aforementioned scene point cloud data. The coordinate system of the aforementioned scene point cloud data is converted to the coordinate system of the aforementioned 3D CAD data. Measurements are performed on the measurement points set in the STL mesh data based on the scene point cloud data which has been moved to the coordinate system of the 3D CAD data. Testing method.

11. In the control unit, STL mesh data is generated from the 3D CAD data of the object. The point cloud acquisition unit acquires scene point cloud data, which is three-dimensional data showing the shape of the aforementioned object. Matching is performed between each point in the master point cloud data generated from the aforementioned STL mesh data and each point in the aforementioned scene point cloud data. The coordinate system of the aforementioned scene point cloud data is converted to the coordinate system of the aforementioned 3D CAD data. Measurements are performed on the measurement points set in the STL mesh data based on the scene point cloud data which has been moved to the coordinate system of the 3D CAD data. A program that performs a process.

Images