Workpiece measurement method, workpiece measurement system, and program

Abstract

To measure the shape and position of a workpiece without requiring preliminary preparation of three-dimensional CAD data, etc.SOLUTION: A workpiece measurement method for measuring the shape and position of a workpiece composed of a plurality of components includes: an acquisition step of acquiring three-dimensional point cloud data of the workpiece; an outline estimation step of estimating one or more boundary frames that each indicate an outline corresponding to each of the plurality of components, using the point cloud data and a condition defined in correspondence with the shape of the workpiece to be measured; and an optimization step of optimizing the one or plurality of boundary frames estimated in the outline estimation step by adjustment of a parameter in accordance with an evaluation function, and identifying a shape of each of the plurality of components.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a workpiece measurement method, a workpiece measurement system, and a program. [Background technology]

[0002] Conventionally, welding robots are used to prepare workpieces to be welded (hereinafter also referred to as workpieces) at predetermined positions and then weld the workpieces. In such cases, for the purpose of automatic welding of the workpieces, it is required to easily grasp the installation state of the workpieces and to reduce labor. When grasping the position of the workpieces, for example, three-dimensional data acquired by a three-dimensional sensor is used.

[0003] As an example of using three-dimensional data, Patent Document 1 discloses a method for identifying two members from three-dimensional CAD data and extracting their shared edges to identify a weld line. Patent Document 2 also discloses a configuration for determining a difference value between reference three-dimensional model data of a reference object measured in advance and three-dimensional data of an actual object during work. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-156566 [Patent Document 2] Patent No. 6917096 Summary of the Invention [Problem to be solved by the invention]

[0005] For example, Patent Document 1 assumes the use of 3D CAD data, and without such data, it is not possible to identify weld lines. Furthermore, Patent Document 2 primarily aims to correct the amount of misalignment of components, and therefore requires the acquisition of 3D model data in advance. Therefore, there is a problem in that it cannot accommodate changes in the size or shape of the target object.

[0006] An object of the present invention is to measure the shape and position of a workpiece without requiring prior preparation such as three-dimensional CAD data. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention has the following configuration: That is, a workpiece measurement method for measuring the shape and position of a workpiece made up of multiple components includes an acquisition step of acquiring three-dimensional point cloud data of the workpiece, an outline estimation step of estimating one or more bounding boxes indicating outline shapes corresponding to the multiple components using the point cloud data and conditions defined corresponding to the shape of the workpiece to be measured, and an optimization step of optimizing the one or more bounding boxes estimated in the outline estimation step by adjusting parameters using an evaluation function to identify the shapes of the multiple components.

[0008] Another aspect of the present invention has the following configuration: A work measurement system that measures the shape and position of a workpiece made up of a plurality of components includes: acquisition means for acquiring three-dimensional point cloud data of the workpiece, outline estimation means for estimating one or more bounding boxes indicating outline shapes corresponding to the plurality of components using the point cloud data and conditions defined corresponding to the shape of the workpiece to be measured, and optimization means for optimizing the one or more bounding boxes estimated by the outline estimation means by adjusting parameters using an evaluation function, and specifying the shapes of the plurality of components.

[0009] Another aspect of the present invention has the following configuration: That is, the program causes a computer to execute an acquisition step of acquiring three-dimensional point cloud data of a workpiece composed of a plurality of components, an outline estimation step of estimating one or more bounding boxes showing outline shapes corresponding to the plurality of components using the point cloud data and conditions defined corresponding to the shape of the workpiece to be measured, and an optimization step of optimizing the one or more bounding boxes estimated in the outline estimation step by adjusting parameters using an evaluation function to identify the shapes of the plurality of components. [Effects of the Invention]

[0010] The present invention makes it possible to measure the shape and position of a workpiece without requiring prior preparation such as three-dimensional CAD data. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an overall outline of a workpiece measurement system according to an embodiment of the present invention; [Figure 2] FIG. 1 is a block diagram showing an example of the functional configuration of an information processing device according to an embodiment of the present invention. [Figure 3] FIG. 2 is a diagram showing an example of point cloud data according to an embodiment of the present invention. [Figure 4] 1 is a schematic diagram showing an example of a workpiece according to an embodiment of the present invention; [Figure 5] 1 is a flowchart of an overall process according to an embodiment of the present invention. [Figure 6] 10 is a flowchart of a general shape estimation process according to an embodiment of the present invention. [Figure 7] FIG. 10 is an explanatory diagram for explaining a general shape estimation process according to an embodiment of the present invention. [Figure 8] FIG. 10 is a graph illustrating the outline estimation process according to the embodiment of the present invention. [Figure 9] 1 is a flowchart of an optimization process according to an embodiment of the present invention. [Figure 10] FIG. 10 is an explanatory diagram for explaining a measurement result according to an embodiment of the present invention. [Figure 11] FIG. 10 is an explanatory diagram for explaining a measurement result according to an embodiment of the present invention. [Figure 12] FIG. 10 is an explanatory diagram for explaining a measurement result according to an embodiment of the present invention. [Figure 13] FIG. 10 is an explanatory diagram for explaining calculation of supplemental information according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. Note that the embodiment described below is one embodiment for explaining the present invention and is not intended to be interpreted as limiting the present invention. Furthermore, not all of the configurations described in each embodiment are necessarily essential configurations for solving the problems of the present invention. Furthermore, in each drawing, the same components are assigned the same reference numerals to indicate corresponding relationships.

[0013] First Embodiment An embodiment of the present invention will be described below with reference to the drawings. In the drawings used in the following description, the three-dimensional coordinate axes indicated by the x-axis, y-axis, and z-axis correspond to each other. In the following description, the plane formed by the x-axis and y-axis directions is defined as the horizontal plane, and the z-axis direction perpendicular to this is defined as the height direction.

[0014] [Measurement system configuration] FIG. 1 shows an overall overview of a workpiece measurement system 1 according to this embodiment. The workpiece measurement system 1 is a system capable of implementing the workpiece measurement method according to this embodiment, and is configured to include an information processing device 100 and a three-dimensional sensor 200. A workpiece 300 to be measured is arranged around the workpiece measurement system 1. In this embodiment, the workpiece 300 may be composed of one or more members. The workpiece measurement system 1 may be used as an integrated configuration with or connected to a welding system configured to include, for example, a welding robot capable of touch sensing. The touch sensing function is a function for grasping the position and shape of the workpiece 300. In this embodiment, an example configuration in which the workpiece measurement system 1 is connected to and capable of cooperation with a welding system 400 having a touch sensing function will be described.

[0015] The information processing device 100 is configured, for example, by a PC (Personal Computer). When the workpiece measurement system 1 according to this embodiment is configured as an integrated unit with the welding system 400, the information processing device 100 may be integrated with a control device for controlling a welding robot (not shown). The information processing device 100 is configured to include a control unit 101, a storage unit 102, a communication unit 103, and a UI (User Interface) unit 104.

[0016] The control unit 101 may be configured using at least one of a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array). The storage unit 102 is configured by a volatile or non-volatile storage device such as an HDD (Hard Disk Drive), a ROM (Read Only Memory), or a RAM (Random Access Memory). The control unit 101 reads and executes various programs stored in the storage unit 102, thereby realizing various functions described below.

[0017] The communication unit 103 is a unit for communicating with external devices and various sensors. Communication by the communication unit 103 may be wired or wireless, and there is no limitation on the communication standard. The UI unit 104 receives operations from a user and displays measurement results. The UI unit 104 may include, for example, a mouse and a keyboard, or may be configured as a touch panel display in which a display unit and an operation unit are integrated. The units within the information processing device 100 are communicably connected by an internal bus (not shown) or the like.

[0018] The three-dimensional sensor 200 is a sensor for acquiring point cloud data as three-dimensional data. For example, a time-of-flight (ToF) camera, a stereo camera, or a light detection and ranging (LiDAR) can be used as the three-dimensional sensor 200. Each of the above sensors has different characteristics, so they may be used depending on the measurement environment and the workpiece 300 to be measured.

[0019] A ToF camera shines laser light onto the object to be measured and measures the reflected laser light with an image sensor to calculate the distance for each pixel. The measurable distance of a ToF camera is on the order of tens of centimeters to several meters. A stereo camera uses multiple images taken by multiple cameras (for example, two) to calculate the distance from the parallax between them. The measurable distance of a stereo camera is on the order of tens of centimeters to several meters. A LiDAR shines laser light onto the surrounding area and calculates the distance by measuring the reflected laser light. The measurable distance of a LiDAR is on the order of tens of centimeters to several meters.

[0020] In this embodiment, an example will be described in which a ToF camera is used as the three-dimensional sensor 200. In this embodiment, the three-dimensional sensor 200 is installed above the workpiece 300 and is capable of capturing images of the workpiece 300 located below. The three-dimensional sensor 200 may be fixed, or may be configured so that its up / down / left / right positions, capturing angle, and capturing conditions can be adjusted depending on the type of image capture.

[0021] [Function Configuration] Fig. 2 is a diagram showing an example of the functional configuration of an information processing device 100 that executes processing related to the workpiece measurement method according to this embodiment. Each component shown in Fig. 2 may be realized by the control unit 101 of the information processing device 100 reading and executing a program stored in the storage unit 102. The information processing device 100 is configured to include a point cloud data acquisition unit 151, a preprocessing unit 152, an outline estimation processing unit 153, a bounding box optimization unit 154, a supplementary information derivation unit 155, a correction processing unit 156, a sensing information acquisition unit 157, and a data management unit 158.

[0022] The point cloud data acquisition unit 151 acquires point cloud data, which is three-dimensional data of the workpiece 300 photographed by the three-dimensional sensor 200. The preprocessing unit 152 performs preprocessing on the acquired point cloud data. The preprocessing here may vary depending on the point cloud data to be used, and examples include filtering, outlier removal, clustering, and coordinate conversion.

[0023] The outline estimation processing unit 153 performs processing to estimate the outline of the workpiece 300 indicated by the point cloud data using a bounding box. The outline here corresponds to the shape of the workpiece 300 approximated based on the point cloud data. The bounding box optimization unit 154 optimizes the bounding box indicating the outline of the workpiece 300 estimated by the outline estimation processing unit 153 to more accurately identify the shape of the workpiece 300. Specific examples of the bounding box according to this embodiment will be described later. The bounding box indicates one or more rectangular or circular boundary frames representing the shapes of one or more components constituting the workpiece 300. The bounding box may be represented as a two-dimensional, i.e., planar, shape, or a three-dimensional, i.e., cubic, shape. Therefore, the bounding box is not limited to a rectangular shape composed of straight lines and may include curved lines. Furthermore, the bounding box and the components constituting the workpiece 300 do not necessarily have a one-to-one relationship; a one-to-many relationship may also exist depending on the shape of the workpiece 300, the direction in which the point cloud data was captured, and other factors. The supplemental information derivation unit 155 derives supplemental information for identifying the three-dimensional position coordinates of the workpiece 300. Examples of the supplemental information will be described later.

[0024] Correction processing unit 156 performs correction processing on the bounding box obtained by bounding box optimization unit 154 to further improve measurement accuracy based on information acquired by a touch sensing function provided by welding system 400. Sensing information acquisition unit 157 acquires measurement information to be used by correction processing unit 156 via the touch sensing function provided by welding system 400. Sensing information acquisition unit 157 may be configured to cause welding system 400 to perform measurements using the touch sensing function. Data management unit 158 ​​holds and manages various data acquired via 3D sensor 200 or the touch sensing function and data generated during measurement operations. By performing touch sensing after determining the shape and position of the workpiece using the workpiece measurement method of this embodiment, the position of the workpiece can be determined more accurately, thereby enabling the position of the weld line to be determined with high accuracy and robot welding to be performed.

[0025] [Point Cloud Data] FIG. 3 shows an example of point cloud data obtained by photographing workpiece 300 with 3D sensor 200. Here, an example of photographing workpiece 300 from above is shown as an oblique view. When viewed from above, workpiece 300 in this example is a cross-shaped workpiece consisting of a diaphragm located in the center and four beam flanges extending on all four sides. A steel joint workpiece is used as an example of workpiece 300.

[0026] There are no particular limitations on the shape, configuration, dimensions, etc. of the workpiece 300. For example, examples of a configuration consisting of a diaphragm and a beam flange include a T-shape consisting of one diaphragm and three beam flanges, and an L-shape or I-shape consisting of one diaphragm and two beam flanges. Further, possible configuration factors include the presence or absence of a step at the connection position between the diaphragm and the beam flange, and an offset at the connection position between the diaphragm and the beam flange.

[0027] FIG. 4 is a diagram illustrating elements of a workpiece 300 to be measured in the workpiece measurement method according to this embodiment. Here, an example of an L-shaped workpiece consisting of one diaphragm 301 and two beam flanges 302 and 303 is shown. Center point 301a indicates the center of the diaphragm 301. Center line 301b is a center line along the x-axis direction that passes through center point 301a of the diaphragm 301. Center line 301c is a center line along the y-axis direction that passes through center point 301a of the diaphragm 301. Center line 302a is a center line along the x-axis direction of beam flange 302 and corresponds to the center position of the short side of beam flange 302. Center line 303a is a center line along the y-axis direction of beam flange 303 and corresponds to the center line of the short side of beam flange 303. Center line 302a and center line 301b are assumed to be parallel, and the difference between them is shown as offset OFF1. The center line 303a and the center line 301c are assumed to be parallel, and the difference therebetween is shown as an offset OFF2. In the following description, the shorter side of the beam flange is also referred to as the width, and the longer side is also referred to as the length.

[0028] In the following explanation, the point cloud data and workpiece shapes described above will be used as examples. However, the present invention is not limited to workpieces with such shapes, and can be applied to workpieces with other shapes.

[0029] [Processing flow] The flow of the workpiece measurement process according to this embodiment will be described below. Fig. 5 is a flowchart showing the overall flow of the workpiece measurement process according to this embodiment. Each step is realized by cooperation between the various parts of the information processing device 100 shown in Fig. 2. Here, to simplify the explanation, the information processing device 100 will be collectively described as the processing subject. Before this processing flow starts, it is assumed that the workpiece 300 is placed in a position where it can be photographed by the 3D sensor 200.

[0030] In step S501, the information processing device 100 acquires point cloud data, which is three-dimensional data captured using the three-dimensional sensor 200. If the workpiece 300 is smaller than a predetermined dimension, the point cloud data may be acquired in a single capturing operation. Alternatively, if the workpiece 300 is larger than a predetermined dimension, multiple point cloud data may be acquired through multiple capturing operations and then integrated. Furthermore, if the workpiece 300 has a predetermined shape, such as a steel beam joint, it is preferable to capture the workpiece 300 from directly above to minimize blind spots. In this step, the information processing device 100 may be configured to adjust the capturing position and angle when capturing images, or the user of the workpiece measurement system 1 may specify these.

[0031] In step S502, the information processing device 100 performs preprocessing on the point cloud data acquired in step S501. Examples of preprocessing include filtering, outlier removal, clustering, and coordinate conversion. Note that the preprocessing in this step may be omitted as long as it is performed according to the configuration of the point cloud data acquired in step S501.

[0032] The filtering process may be, for example, a process of resampling the points included in the point cloud data at regular intervals using a known voxel grid filter or the like to maintain a constant point cloud density per given volume. The outlier removal process may be a process of removing outliers that could reduce measurement accuracy. Outliers may be identified, for example, from statistical information such as the mean and variance of adjacent point clouds, or from the number of adjacent point clouds within a given radius. The clustering process may be, for example, a process of dividing the points included in the point cloud data into multiple groups based on distance or the like, and removing groups with a number of points below a given threshold, thereby removing point clouds other than those representing the shape of the workpiece 300. The coordinate conversion process converts the coordinate system of the 3D sensor 200 into a predetermined coordinate system based on the imaging position and imaging angle of the 3D sensor 200. The predetermined coordinate system may be, for example, a coordinate system used in the touch sensing function, or a coordinate system in which the origin and coordinate axes are defined with the surface on which the workpiece 300 is placed as the xy plane, as shown in FIG. 1. The parameters required for the transformation of the coordinate system may be derived in advance by a calibration process or the like.

[0033] In step S503, the information processing device 100 performs a general shape estimation process using the point cloud data processed in step S502. Details of this process will be described later with reference to Figures 6 to 8. As a result of this process, one or more bounding boxes that indicate the general shape of the workpiece 300 are obtained.

[0034] In step S504, the information processing apparatus 100 performs an optimization process to correct the bounding box obtained in step S503. Details of this process will be described later with reference to FIG.

[0035] In step S505, the information processing device 100 derives supplemental information for identifying the position coordinates of the workpiece 300 in three-dimensional coordinates. The supplemental information includes, for example, height information of each part constituting the workpiece 300. For example, as shown in FIG. 3, the top surfaces of the diaphragm and beam flange constituting the workpiece 300 are not necessarily horizontal, and the height may vary depending on the position. Therefore, height information may be included as supplemental information to measure the shape of the workpiece 300 more accurately. In addition, the angle between the surface of each part and the horizontal plane may be included as supplemental information.

[0036] FIG. 13 is an explanatory diagram for explaining the calculation of a parameter in the z-axis direction, i.e., height, as supplementary information. In this example, similar to FIG. 3, the workpiece 300 is a cross-shaped workpiece composed of one diaphragm and four beam flanges. Regarding the diaphragm, the measurement accuracy of the point cloud data is assumed to be low in the peripheral area, and the height of the diaphragm is calculated using height information within region 1301, which excludes the point cloud data of the peripheral area located within a predetermined range from the boundary. Regarding the beam flanges, the measurement accuracy of the point cloud data is also assumed to be low in the peripheral area. Furthermore, height information of positions close to the diaphragm is used. Therefore, when calculating the height of each beam flange, the height information of the point cloud data in region 1302 is used.

[0037] In step S506, the information processing device 100 performs correction processing based on the information acquired by the touch sensing function. The operations in this step may be performed after the information processing device 100 determines whether or not to perform touch sensing operation and correction processing using the touch sensing function based on the measurement results obtained by the processing up to this point. Alternatively, the user of the work measurement system 1 may be configured to specify whether or not to perform this step. Therefore, this step may be omitted. Then, this processing flow ends.

[0038] (Outline estimation processing) Fig. 6 is a flowchart of the outline estimation process according to this embodiment, which corresponds to step S503 in Fig. 5. First, an overview of the outline estimation process will be described with reference to Figs.

[0039] FIG. 7 shows point cloud data obtained when the workpiece 300 is photographed from directly above by the 3D sensor 200. In other words, the point cloud data shown in FIG. 3 is converted so as to be projected onto a two-dimensional plane. In this example, the workpiece 300 has a cross shape. In the general shape estimation process, general shape estimation is performed sequentially from multiple directions defined according to the shape of the workpiece 300 to be measured. For example, if the workpiece 300 is cross-shaped, operations are performed in four directions: from right to left and left to right along the x-axis, and from top to bottom and bottom to top along the y-axis. FIG. 7 shows an example of left-to-right scanning along the x-axis. Furthermore, a region of interest (ROI) 701 is set, and general shape estimation is performed while shifting the ROI 701. FIG. 7 shows an example in which the width of the left-hand beam flange in the figure, of the four beam flanges that make up the workpiece 300, is estimated as the general shape from left to right along the x-axis.

[0040] FIG. 8 is a diagram for explaining the general shape estimation process shown in FIG. 7 and is a graph illustrating an example of the results of the general shape estimation in FIG. 7. In FIG. 8, the horizontal axis represents the detection results in the first axis direction (the x-axis direction in the example of FIG. 7), and the vertical axis represents the detection results in the second axis direction (the y-axis direction in the example of FIG. 7). In the measurement example of FIG. 7, the horizontal axis corresponds to the length of the beam flange, and the vertical axis corresponds to the width of the beam flange. Furthermore, on the horizontal axis, the position at which point cloud detection begins as the ROI 701 is moved is indicated as "0." The values ​​on each axis are merely examples and may vary depending on the size of the workpiece 300 being measured. By identifying point cloud data corresponding to the beam flange as shown in FIG. 7, the width of the beam flange constituting the workpiece 300, i.e., the general shape of the workpiece 300, can be estimated. In this example, the width of the beam flange can be estimated as "200." Furthermore, as the estimation process continues, the width value increases sharply at the length position of "500." As a result, the length of the beam flange can be estimated as "500."

[0041] As described above, in the example of FIG. 7, outline estimation is performed by scanning point cloud data from four directions. In this case, the outline estimation direction may be switched when a sudden change in measurement value occurs, as shown in FIG. 8. Alternatively, the outline estimation process may be performed on all point cloud data from all directions. The following describes a mode in which the outline estimation direction is switched when the measurement value changes by a threshold value or more.

[0042] The bounding box information obtained by general shape estimation includes, for example, the following items. Note that depending on the shape of the workpiece 300 to be measured, the user may specify some of the information listed below. Shape: Rectangle, circle, cube, etc. Size: If the shape is rectangular, the length of the long side, the length of the short side, the center coordinate, the angle, etc. Count: The number of bounding boxes per shape Constraints: Connections and positional relationships between bounding boxes

[0043] Returning to Fig. 6, the flowchart corresponding to the processing in Fig. 7 will be described. As a premise of this processing flow, conditions according to the shape of the workpiece 300 are specified when estimating the general shape of the workpiece 300 in order to obtain the bounding box information as described above. For example, if the workpiece 300 has a cross shape as shown in Fig. 3 and is a steel-frame joint workpiece, the conditions for general shape estimation include the following:

[0044] Shape: Rectangle Size: Varies depending on material Quantity: 1 diaphragm + 2 to 4 beam flanges Constraint: The short side of the beam flange must be in contact with one side of the diaphragm, and different beam flanges must not be in contact with each other.

[0045] Under the estimation conditions described above, the general shape of the workpiece 300 is estimated. The estimation conditions for general shape estimation are predetermined according to the shape of the workpiece 300. Furthermore, the estimation conditions are not limited to the above estimation conditions, and any estimation conditions can be defined. The conditions for performing the general shape estimation described above may be predetermined according to the type of workpiece 300 to be measured.

[0046] Furthermore, in order to estimate the general shape of the joints of the members constituting the workpiece 300, it is necessary to observe the workpiece 300 from above, i.e., from the direction along the z-axis in Fig. 1, etc. Therefore, when using point cloud data such as that shown in Fig. 3, the point cloud data is rotated and converted to a bird's-eye view from above as shown in Fig. 7. This conversion may be performed in the preprocessing in step S502 of Fig. 5.

[0047] The loop processing of steps S601 to S611 is repeated for the point cloud data at each predetermined angle around the z-axis on the xy plane shown in Fig. 1 etc. The predetermined angle and the start angle here are assumed to be predefined, and for example, the predetermined angle may be 5 degrees and the start angle may be an angle (0 degrees) based on the direction along the x-axis on the xy plane.

[0048] In step S602, the outline estimation processing unit 153 sets an ROI 701 on the xy plane of the point cloud data. The ROI 701 set here corresponds to the start position of outline estimation, and multiple ROIs may be set according to the number of scanning directions, i.e., the number of scanning directions of the point cloud data. The size of the ROI 701 is not particularly limited, and may be a fixed size or may be specified according to the image size of the point cloud data, for example.

[0049] Next, the loop process of steps S603 to S609 is repeated for each predetermined direction on the xy plane for the point cloud data. Here, it is assumed that the point cloud data is scanned in four scanning directions: the positive x-axis direction, the positive y-axis direction, the negative x-axis direction, and the negative y-axis direction. The direction and number of scanning directions may be specified based on the above estimation conditions.

[0050] In step S604, the outline estimation processing unit 153 calculates the width of the point cloud in the set ROI 701.

[0051] In step S605, the outline estimation processing unit 153 determines whether the difference between the width of the point cloud calculated in step S604 and the width calculated at the previous position of the ROI 701, i.e., the amount of change, is equal to or greater than a threshold. The threshold here is assumed to be specified in advance. Note that the determination may be made using the rate of change instead of the amount of change. If the amount of change is equal to or greater than the threshold (YES in step S605), the process of the outline estimation processing unit 153 proceeds to step S607. On the other hand, if the amount of change is less than the threshold (NO in step S605), the process of the outline estimation processing unit 153 proceeds to step S606.

[0052] In step S606, the outline estimation processing unit 153 translates the ROI 701 in the scanning direction. The amount of movement of the ROI 701 here is assumed to be predetermined based on the size of the ROI 701, etc. Then, the process of the outline estimation processing unit 153 returns to step S604 and the process is repeated.

[0053] In step S607, the outline estimation processing unit 153 sets the value immediately before the position where the amount of change is greater than or equal to the threshold as the width of the beam flange, and sets the length from the position where the point cloud is detected to the position where the amount of change is greater than or equal to the threshold as the length of the beam flange.

[0054] In step S608, if the length of the beam flange set in step S607 is equal to or less than the threshold value, the general shape estimation processing unit 153 sets that there is no beam flange in the direction in question. That is, it sets the number of beam flanges.

[0055] In step S609, the outline estimation processing unit 153 determines whether scanning from all scanning directions has been completed. If scanning from all scanning directions has not been completed, the process switches to an unprocessed scanning direction and repeats the processing from step S604 onwards. On the other hand, if scanning from all scanning directions has been completed, the processing of the outline estimation processing unit 153 proceeds to step S610.

[0056] In step S610, the general shape estimation processing unit 153 calculates the angle formed by the center line of each beam flange and the x-axis or y-axis on the xy plane.

[0057] In step S611, the outline estimation processing unit 153 determines whether or not the calculation process for all rotation angles has been completed. If the calculation process for all rotation angles has not been completed, the image is rotated to the next angle, and the process from step S602 onwards is repeated. On the other hand, if the calculation process for all rotation angles has been completed, the process of the outline estimation processing unit 153 proceeds to step S612.

[0058] In step S612, the general shape estimation processing unit 153 determines the rotation position at which the average angle formed by the center line of each beam flange constituting the workpiece 300 and the x-axis or y-axis is minimum as the installation angle of the workpiece 300. In other words, the rotation angle of the point cloud data around the z-axis is determined so that the x-axis, y-axis, and each beam flange are in a parallel positional relationship on the xy plane.

[0059] In step S613, the general shape estimation processing unit 153 calculates the dimensions of the diaphragms that make up the workpiece 300 from the width and length of the entire point cloud and the calculated lengths of each beam flange. The diaphragm dimensions can be calculated based on the estimation conditions described above. This makes it possible to generate one or more bounding boxes that indicate the general shape of the entire workpiece 300. For example, in the case of a workpiece 300 having the shape shown in FIG. 7, a total of five bounding boxes can be generated, one for each diaphragm and four beam flanges. In other words, the general shape of each member that makes up the workpiece 300 is identified by the parameters of the bounding boxes. Then, this processing flow ends.

[0060] (Bounding box optimization process) FIG. 9 is a flowchart of the bounding box optimization process according to this embodiment, which corresponds to step S504 in FIG.

[0061] In step S901, bounding box optimization unit 154 sets the parameters of each bounding box obtained by the outline estimation process in step S503 of Fig. 5 as initial values. In other words, the parameters obtained by the outline estimation process are used as references, and the measurement accuracy is improved by optimization processing.

[0062] In step S902, the bounding box optimization unit 154 calculates an evaluation value using an evaluation function. In this embodiment, the following formula (1) is used as the evaluation function, which indicates a weighted linear sum based on the number of points in the point cloud and the point cloud density. F(x)=W N N(x)+W D D(x) (1) W N ,W D : weighting factor x: Optimization parameter N(x): Number of points in the bounding box D(x): Point cloud density within the bounding box The optimization parameter x may be, for example, the position, angle, width, length, etc. of the bounding box. More specifically, in the case of the example of the workpiece 300 configured as shown in Fig. 4, the angle of the workpiece 300 around the z-axis, the center coordinates of the diaphragm, the width, length, offset, etc. of the beam flange may be used as the optimization parameter.

[0063] In step S903, the bounding box optimization unit 154 calculates the amount of change in the evaluation value based on the initial value set in step S901 and the evaluation value calculated in step S902. In this embodiment, the amount of change in the evaluation value is calculated while changing each optimization parameter from the initial value at a predetermined interval.

[0064] In step S904, the bounding box optimization unit 154 updates the parameters based on the gradient vector calculated in step S903. Here, the parameters may be updated using, for example, a known method such as the steepest descent method.

[0065] In step S905, bounding box optimization unit 154 determines whether, as a result of updating the parameters, the amount of change has converged within a predetermined range or the number of updates has reached a predetermined threshold. If the amount of change has converged within the predetermined range or the number of updates has reached a predetermined threshold (YES in step S905), this processing flow ends. Otherwise (NO in step S905), the processing of bounding box optimization unit 154 returns to step S902 and the processing is repeated.

[0066] [Explanation of processing results] 10 to 12 are diagrams showing measurement results obtained by the workpiece measurement method according to this embodiment. Here, a method using a well-known OBB (Oriented Bounding Box) will be described as an example of a conventional method to be compared with the workpiece measurement method according to this embodiment.

[0067] FIG. 10 shows an example of a bounding box as a measurement result using point cloud data 1000 of a workpiece. The point cloud data 1000 contains a certain amount of noise around the periphery. Bounding box 1001 shows the measurement result obtained using a conventional method, while bounding box 1002 shows the measurement result obtained using the workpiece measurement method according to this embodiment. The conventional method is affected by noise and shows a larger range than the point cloud representing the actual workpiece. On the other hand, the workpiece measurement method according to this embodiment obtains a bounding box 1002 that is closer to the shape of the actual workpiece.

[0068] FIG. 11 shows an example of a bounding box as a measurement result using point cloud data 1100 of a workpiece. In the point cloud data 1100, the workpiece was not photographed correctly using a 3D camera, resulting in a partial defect 1101. Bounding box 1102 shows the measurement result obtained using a conventional method, while bounding box 1103 shows the measurement result obtained using the workpiece measurement method according to this embodiment. With the conventional method, the influence of defect 1101 prevented the spindle from being correctly identified, resulting in errors in angle and size compared to the point cloud representing the actual workpiece. In contrast, the workpiece measurement method according to this embodiment suppresses the influence of defect 1101, resulting in a bounding box 1103 that is closer to the angle and size of the actual workpiece.

[0069] FIG. 12 shows an example of a bounding box as a measurement result using point cloud data 1200 of a workpiece. In point cloud data 1200, the workpiece has accessories attached in two locations, and point clouds 1201 corresponding to the accessories are included. Bounding box 1202 represents the measurement result obtained using a conventional method, while bounding box 1203 represents the measurement result obtained using the workpiece measurement method according to this embodiment. The conventional method is affected by point cloud 1201, resulting in a larger range than the point cloud representing the actual workpiece. On the other hand, the workpiece measurement method according to this embodiment removes point cloud 1201 that is smaller than a predetermined size to suppress its influence, resulting in a bounding box 1203 that more closely resembles the shape of the actual workpiece.

[0070] As described above, this embodiment makes it possible to measure the shape and position of a workpiece without the need for prior preparation of 3D CAD data, etc. Furthermore, it makes it possible to grasp the shape of the workpiece more accurately than conventional methods.

[0071] <Other embodiments> The workpiece measurement method described in the above embodiment can be applied to a welding system including a welding robot, which makes it possible to automatically extract a weld line for a target workpiece based on, for example, bounding box information.

[0072] In addition, although the above embodiment shows an example in which the shape of the workpiece is measured from a viewpoint above the workpiece, it may also be configured to measure from multiple directions, thereby suppressing the influence of blind spots and enabling more accurate measurements.

[0073] This embodiment can also be realized by supplying a program or application for realizing the functions of one or more of the above-mentioned embodiments to a system or device using a network or storage medium, etc., and having one or more processors in the computer of that system or device read and execute the program.

[0074] The present embodiment may be realized by a circuit that realizes one or more functions, such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0075] As described above, the present specification discloses the following: (1) A workpiece measurement method for measuring the shape and position of a workpiece made up of multiple members, comprising: an acquisition step of acquiring three-dimensional point cloud data of the workpiece; an outline estimation step of estimating one or more boundary frames indicating outline shapes corresponding to the plurality of members using conditions defined in accordance with the shape of the workpiece to be measured and the point cloud data; an optimization step of optimizing the one or more bounding boxes estimated in the general shape estimation step by adjusting parameters using an evaluation function, and specifying shapes of the plurality of members; A workpiece measurement method having the above construction. According to this configuration, it is possible to measure the shape and position of a workpiece without the need for prior preparation such as three-dimensional CAD data.

[0076] (2) The workpiece measurement method according to (1), wherein the conditions include any of the shape, size, number, and constraints of the bounding box corresponding to the workpiece. According to this configuration, it is possible to estimate the bounding box of a workpiece based on any one of the shape, size, number, and constraints as a condition corresponding to the workpiece.

[0077] (3) The workpiece measurement method according to (1) or (2), wherein the evaluation function is a weighted linear sum based on the number of points in the point cloud included in the bounding box and the point cloud density. According to this configuration, it is possible to optimize the bounding box based on a weighted linear sum of the number of points in the point cloud within the bounding box and the point cloud density.

[0078] (4) The workpiece measurement method according to any one of (1) to (3), wherein in the optimization step, at least one of the parameters of the size, position, and angle of the bounding box is optimized. This configuration makes it possible to optimize at least one of the dimensions, position, and angle of the bounding box.

[0079] (5) The workpiece measurement method according to any one of (1) to (4), wherein in the general shape estimation step, the boundary frame is estimated by scanning the point cloud data from a plurality of directions. According to this configuration, it is possible to estimate the general shape of the workpiece with higher accuracy.

[0080] (6) The workpiece measurement method according to any one of (1) to (5), wherein in the rough shape estimation step, the rough shape is estimated by projecting the point cloud data onto a two-dimensional plane. According to this configuration, it is possible to accurately estimate the shape of the point cloud data projected onto a two-dimensional plane from a desired direction.

[0081] (7) The workpiece measurement method according to (6), further comprising a deriving step of deriving positions in an axial direction perpendicular to the two-dimensional plane, corresponding to the plurality of members. According to this configuration, it is possible to further derive information about the height direction of each member constituting the workpiece relative to the two-dimensional plane.

[0082] (8) A workpiece measurement method according to any one of (1) to (7), wherein the bounding box is configured to include straight lines or curved lines. According to this configuration, it is possible to specify the shape of a workpiece using a bounding box of any shape.

[0083] (9) A workpiece measurement method according to any one of (1) to (8), wherein the bounding box is displayed in two dimensions or three dimensions. According to this configuration, it is possible to identify the shape of the workpiece using a two-dimensional or three-dimensional bounding box.

[0084] (10) A work measurement method described in any one of (1) to (9), further comprising a correction step of correcting the one or more bounding boxes optimized in the optimization step using the touch sensing measurement results for the work. According to this configuration, it is possible to further improve the measurement accuracy by using the results of touch sensing.

[0085] (11) A work measurement system for measuring the shape and position of a workpiece made up of multiple members, an acquisition means for acquiring three-dimensional point cloud data of the workpiece; an outline estimation means for estimating one or more boundary frames that indicate outlines corresponding to the plurality of components, using conditions defined in accordance with the shape of the workpiece to be measured and the point cloud data; an optimization means for optimizing the one or more bounding boxes estimated by the general shape estimation means by adjusting parameters using an evaluation function, and specifying the shapes of the plurality of components; A workpiece measurement system having According to this configuration, it is possible to measure the shape and position of a workpiece without the need for prior preparation such as three-dimensional CAD data.

[0086] (12) To the computer, an acquisition step of acquiring three-dimensional point cloud data of a workpiece composed of a plurality of members; an outline estimation step of estimating one or more boundary frames indicating outline shapes corresponding to the plurality of members using conditions defined in accordance with the shape of the workpiece to be measured and the point cloud data; an optimization step of optimizing the one or more bounding boxes estimated in the general shape estimation step by adjusting parameters using an evaluation function, and specifying shapes of the plurality of members; A program to execute. According to this configuration, it is possible to measure the shape and position of a workpiece without the need for prior preparation such as three-dimensional CAD data. [Explanation of symbols]

[0087] 1...Workpiece measurement system 100...Information processing device 101...Control unit 102...Storage section 103…Communications Department 104...UI section 151...Point cloud data acquisition unit 152...Preprocessing section 153...Outline estimation processing unit 154...Bounding box optimization section 155…Supplementary information derivation part 156...Correction processing unit 157...Sensing information acquisition unit 158...Data Management Department 200...3D camera 300...Work 400...Welding system

Claims

1. A workpiece measurement method for measuring the shape and position of a workpiece made up of a plurality of members, comprising: an acquisition step of acquiring three-dimensional point cloud data of the workpiece; an outline estimation step of estimating one or more boundary frames indicating outline shapes corresponding to the plurality of members using conditions defined in accordance with the shape of the workpiece to be measured and the point cloud data; an optimization step of optimizing the one or more bounding boxes estimated in the general shape estimation step by adjusting parameters using an evaluation function, and specifying shapes of the plurality of members; a correction step of correcting the one or more bounding boxes optimized in the optimization step using the measurement results of touch sensing on the workpiece.

2. The workpiece measurement method according to claim 1 , wherein the conditions include any one of the shape, size, number, and constraints of a bounding box corresponding to the workpiece.

3. The workpiece measurement method according to claim 1 or 2, wherein the evaluation function is a weighted linear sum based on the number of points in the point cloud included in the bounding box and the point cloud density.

4. The workpiece measurement method according to claim 1 , wherein the optimization step optimizes at least one parameter of the size, position, and angle of the bounding box.

5. The workpiece measurement method according to claim 1 , wherein in the general shape estimation step, the boundary frame is estimated by scanning the point cloud data from a plurality of directions.

6. The workpiece measurement method according to claim 1 , wherein in the general shape estimation step, the general shape is estimated by projecting the point cloud data onto a two-dimensional plane.

7. The workpiece measuring method according to claim 6 , further comprising a deriving step of deriving positions in an axial direction perpendicular to the two-dimensional plane, corresponding to the plurality of members.

8. The workpiece measurement method according to claim 1 , wherein the boundary frame is configured to include straight lines or curved lines.

9. The workpiece measurement method according to claim 1 , wherein the bounding box is displayed in two or three dimensions.

10. A workpiece measurement system that measures the shape and position of a workpiece made up of multiple members, an acquisition means for acquiring three-dimensional point cloud data of the workpiece; an approximate shape estimation means for estimating one or more boundary frames indicating approximate shapes corresponding to the plurality of members using conditions defined in accordance with the shape of the workpiece to be measured and the point cloud data; an optimization means for optimizing the one or more bounding boxes estimated by the general shape estimation means by adjusting parameters using an evaluation function, and specifying shapes of the plurality of components; a correction means for correcting the one or more bounding boxes optimized by the optimization means using a measurement result of touch sensing on the workpiece; A workpiece measurement system having

11. On the computer, an acquisition step of acquiring three-dimensional point cloud data of a workpiece composed of a plurality of members; an outline estimation step of estimating one or more boundary frames indicating outline shapes corresponding to the plurality of members using conditions defined in accordance with the shape of the workpiece to be measured and the point cloud data; an optimization step of optimizing the one or more bounding boxes estimated in the general shape estimation step by adjusting parameters using an evaluation function, and specifying shapes of the plurality of members; a correction step of correcting the one or more bounding boxes optimized in the optimization step using a measurement result of touch sensing on the workpiece; A program to execute.

Images