Workpiece measurement method, workpiece measurement system, and program

Abstract

Provided is a workpiece measurement method, a workpiece measurement system, and a program capable of measuring the shape and position of a workpiece without preparing three-dimensional CAD data or the like in advance. A workpiece measurement method of measuring the shape and position of a workpiece composed of a plurality of components includes: an acquisition step of acquiring three-dimensional point cloud data of the workpiece; a profile inference step of inferring one or more bounding boxes representing profiles corresponding to the plurality of components using conditions defined in correspondence with the shape of the workpiece to be measured and the point cloud data; and an optimization step of optimizing the one or more bounding boxes inferred in the profile inference step by adjusting parameters using an evaluation function to determine the shape of the plurality of components.

Description

Technical Field

[0001] This invention relates to a workpiece measurement method, a workpiece measurement system, and a program. Background Technology

[0002] Traditionally, welding robots are used to prepare components (hereinafter referred to as workpieces) at predetermined locations for welding. In such cases, for the purpose of automated welding of workpieces, it is necessary to easily control the workpiece's position and achieve labor-saving operation. When controlling the workpiece's position, for example, three-dimensional data obtained using three-dimensional sensors is employed.

[0003] As an example of using three-dimensional data, Patent Document 1 discloses the following: two components are determined based on three-dimensional CAD data, their shared edges are extracted, and the weld line is determined. Furthermore, Patent Document 2 discloses a structure that calculates the difference value based on reference three-dimensional model data of a reference object measured in advance and the three-dimensional data of the actual object during operation.

[0004] Prior art literature

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2018-156566

[0007] Patent Document 2: Japanese Patent No. 6917096 Summary of the Invention

[0008] The problem the invention aims to solve

[0009] For example, Patent Document 1 relies on 3D CAD data; without such data, the weld line cannot be determined. Furthermore, Patent Document 2, which primarily aims to correct component offsets, requires prior access to 3D model data. Therefore, it suffers from limitations in handling variations in the size and shape of the object.

[0010] The purpose of this invention is to determine the shape and position of a workpiece without prior preparation of 3D CAD data.

[0011] means for solving problems

[0012] To address the aforementioned problems, the present invention has the following structure. Specifically, a workpiece measurement method for measuring the shape and position of a workpiece composed of multiple components includes: an acquisition step, in which three-dimensional point group data of the workpiece is acquired; a probabilistic inference step, in which one or more bounding boxes representing probabilistic shapes corresponding to the multiple components are inferred using conditions defined corresponding to the shape of the workpiece being measured and the point group data; and an optimization step, in which the one or more bounding boxes inferred in the probabilistic inference step are optimized by adjusting parameters using an evaluation function, thereby determining the shape of the multiple components.

[0013] Another aspect of the present invention has the following structure. That is, a workpiece measurement system for measuring the shape and position of a workpiece composed of multiple components includes: an acquisition unit that acquires three-dimensional point group data of the workpiece; a probabilistic inference unit that uses conditions defined corresponding to the shape of the workpiece and the point group data to infer one or more bounding boxes representing probabilistic shapes corresponding to the multiple components; and an optimization unit that optimizes the one or more bounding boxes inferred by the probabilistic inference unit by adjusting parameters using an evaluation function, thereby determining the shape of the multiple components.

[0014] Another aspect of the present invention has the following structure. That is, the program causes a computer to perform the following steps: an acquisition step, in which three-dimensional point group data of a workpiece composed of multiple components is acquired; a probabilistic inference step, in which one or more bounding boxes representing the probabilistic shape corresponding to the multiple components are inferred using conditions specified corresponding to the shape of the workpiece being measured and the point group data; and an optimization step, in which the one or more bounding boxes inferred in the probabilistic inference step are optimized by adjusting parameters using an evaluation function, thereby determining the shape of the multiple components.

[0015] Invention Effects

[0016] According to the present invention, the shape and position of a workpiece can be determined without prior preparation of 3D CAD data. Attached Figure Description

[0017] Figure 1 This is a block diagram illustrating the overall outline of a workpiece measurement system according to an embodiment of the present invention.

[0018] Figure 2 This is a block diagram illustrating an example of the functional structure of an information processing apparatus according to an embodiment of the present invention.

[0019] Figure 3 This is an example diagram illustrating a point group data according to an embodiment of the present invention.

[0020] Figure 4 This is a schematic diagram illustrating an example of a workpiece according to an embodiment of the present invention.

[0021] Figure 5 This is a flowchart of the overall process according to one embodiment of the present invention.

[0022] Figure 6 This is a flowchart of a probabilistic deduction process according to an embodiment of the present invention.

[0023] Figure 7 This is an explanatory diagram illustrating the probabilistic deduction process of one embodiment of the present invention.

[0024] Figure 8 This is a coordinate graph used to illustrate the probabilistic deduction process of one embodiment of the present invention.

[0025] Figure 9 This is a flowchart of an optimization process according to one embodiment of the present invention.

[0026] Figure 10 This is an explanatory diagram illustrating the measurement results of one embodiment of the present invention.

[0027] Figure 11 This is an explanatory diagram illustrating the measurement results of one embodiment of the present invention.

[0028] Figure 12 This is an explanatory diagram illustrating the measurement results of one embodiment of the present invention.

[0029] Figure 13 This is an explanatory diagram illustrating the calculation of supplementary information for explaining one embodiment of the present invention.

[0030] Explanation of reference numerals in the attached figures

[0031] 1…Workpiece measurement system;

[0032] 100…information processing device;

[0033] 101…Control Department;

[0034] 102… Storage Department;

[0035] 103…Ministry of Communications;

[0036] 104…UI Department;

[0037] 151… point group data acquisition department;

[0038] 152…Pre-processing section;

[0039] 153…Probability Inference Processing Department;

[0040] 154…Bounding Box Optimization Department;

[0041] 155…Supplementary Information Export Department;

[0042] 156…Correction Processing Department;

[0043] 157…Sensing information acquisition unit;

[0044] 158…Data Management Department;

[0045] 200…3D camera;

[0046] 300…workpiece;

[0047] 400… Welding system. Detailed Implementation

[0048] Hereinafter, embodiments for carrying out the present invention will be described with reference to the accompanying drawings. It should be noted that the embodiments described below are illustrative of one embodiment of the present invention and are not intended to limit the interpretation of the invention. Furthermore, not all structures described in each embodiment are necessarily necessary to solve the problems of the present invention. In addition, in each figure, the same reference numerals are used to indicate the correspondence of the same constituent elements.

[0049] <First Implementation Method>

[0050] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. It should be noted that in the figures used in the following description, the three-dimensional coordinate axes represented by the x-axis, y-axis, and z-axis correspond to each other. In the following description, the plane including the x-axis direction and the y-axis direction is defined as a horizontal plane, and the z-axis direction, which is orthogonal to it, is defined as the height direction.

[0051] [Structure of the Measurement System]

[0052] Figure 1 This section shows an overall overview of the workpiece measurement system 1 according to this embodiment. The workpiece measurement system 1 is a system capable of incorporating the workpiece measurement method of this embodiment, and is configured to include an information processing device 100 and a three-dimensional sensor 200. Furthermore, a workpiece 300, which is the object of measurement, is arranged around the workpiece measurement system 1. In this embodiment, the workpiece 300 may be composed of one or more components. The workpiece measurement system 1 may be used, for example, as a structure integrated with or connected to a welding system, which is configured to include a welding robot capable of touch sensing. The touch sensing function is used to determine the position and shape of the workpiece 300. In this embodiment, a structure in which the workpiece measurement system 1 is connected to a welding system 400 with touch sensing function and can cooperate is described as an example.

[0053] The information processing device 100 is configured, for example, as a PC (Personal Computer). It should be noted that, in the case where the workpiece measurement system 1 of this embodiment is integrated with the welding system 400, the information processing device 100 may also be integrated with a control device for controlling a welding robot (not shown). The information processing device 100 is configured to include a control unit 101, a storage unit 102, a communication unit 103, and a UI (User Interface) unit 104.

[0054] The control unit 101 may be configured using at least one of a CPU (Central Processing Unit), GPU (Graphical Processing Unit), MPU (Micro Processing Unit), DSP (Digital Signal Processor), or FPGA (Field Programmable Gate Array). The storage unit 102 may be configured using volatile or non-volatile storage devices such as HDD (Hard Disk Drive), ROM (Read Only Memory), or RAM (Random Access Memory). The various functions described later are achieved by the control unit 101 reading and executing the various programs stored in the storage unit 102.

[0055] The communication unit 103 is used for communication with external devices and various sensors. Communication with the communication unit 103 can be wired or wireless, and the communication standard is not limited. The user interface (UI) unit 104 receives user input or displays measurement results. The UI unit 104 may include, for example, a mouse or keyboard, or a touch panel display that integrates the display and operation units. The various components within the information processing device 100 are connected communicatively via an internal bus (not shown).

[0056] The 3D sensor 200 is a sensor used to acquire point group data as 3D data. For example, a ToF (Time of Flight) camera, a stereo camera, or LiDAR (Light Detection and Ranging) can be used as the 3D sensor 200. Each of these sensors has different characteristics, therefore, they can be used separately depending on the measurement environment and the workpiece 300 being measured.

[0057] Time-of-Flight (ToF) cameras calculate distance by shining a laser beam onto the object and measuring the reflected laser light using an imaging element, down to the pixel level. The measurable distance of a ToF camera is in the range of tens of centimeters to several meters. Stereo cameras use multiple images captured by multiple cameras (e.g., two) and calculate distance based on their parallax. The measurable distance of a stereo camera is also in the range of tens of centimeters to several meters. LiDAR calculates distance by shining a laser beam around the object and measuring the reflected laser light. The measurable distance of a LiDAR camera is also in the range of tens of centimeters to tens of meters.

[0058] In this embodiment, an example of using a ToF camera as a 3D sensor 200 will be described. In this embodiment, the 3D sensor 200 is positioned above the workpiece 300 and is capable of capturing images of the workpiece 300 located below it. It should be noted that the 3D sensor 200 can be fixed or configured to adjust its position (up, down, left, right), shooting angle, and shooting conditions according to the shooting situation.

[0059] [Functional Structure]

[0060] Figure 2 This is a diagram illustrating an example of the functional structure of an information processing device 100 that performs the workpiece measurement method of this embodiment. Figure 2 Each of the shown parts can be implemented by reading and executing the program stored in the storage unit 102 through the control unit 101 of the information processing device 100. The information processing device 100 is configured to include a point group data acquisition unit 151, a preprocessing unit 152, a probabilistic inference processing unit 153, a bounding box optimization unit 154, a supplementary information export unit 155, a correction processing unit 156, a sensing information acquisition unit 157, and a data management unit 158.

[0061] The point group data acquisition unit 151 acquires point group data, which is the three-dimensional data of the workpiece 300 captured by the three-dimensional sensor 200. The preprocessing unit 152 preprocesses the acquired point group data. The preprocessing can vary depending on the point group data used, and may include, for example, filtering, outlier removal, clustering, coordinate transformation, etc.

[0062] The probabilistic inference processing unit 153 performs a process of inferring the probabilistic shape of the workpiece 300 represented by point group data using bounding boxes. Here, the probabilistic shape is equivalent to the shape of the workpiece 300 obtained by approximating it based on the point group data. The bounding box optimization unit 154 optimizes the bounding box representing the probabilistic shape of the workpiece 300 inferred by the probabilistic inference processing unit 153, thereby determining the shape of the workpiece 300 with higher accuracy. Specific examples of the bounding box in this embodiment will be described later, but one or more bounding boxes, such as rectangles or circles, used to represent the shapes of one or more components constituting the workpiece 300 are shown. The bounding box can be represented by a two-dimensional, planar shape, or by a three-dimensional, solid shape. Therefore, the bounding box is not limited to a rectangular shape composed of straight lines, and may also include curves in a portion thereof. Furthermore, the bounding box and the components constituting the workpiece 300 are not necessarily in a one-to-one relationship; depending on the shape of the workpiece 300, the shooting direction of the point group data, etc., a one-to-many relationship may also be present. The supplementary information derivation unit 155 derives supplementary information for determining the position coordinates of the workpiece 300 in three-dimensional coordinates. Examples of supplementary information will be described later.

[0063] The calibration processing unit 156 performs calibration processing on the bounding box obtained by the bounding box optimization unit 154, based on information obtained through the touch sensing function provided by the welding system 400, to further improve measurement accuracy. The sensing information acquisition unit 157 acquires measurement information for use in the calibration processing unit 156 via the touch sensing function provided by the welding system 400. The sensing information acquisition unit 157 can be a structure that enables the welding system 400 to perform measurements based on the touch sensing function. The data management unit 158 ​​stores and manages various data acquired via the three-dimensional sensor 200, the touch sensing function, and data generated during the measurement operation. By using the workpiece measurement method of this embodiment to determine the shape and position of the workpiece, touch sensing is performed, thereby enabling higher precision determination of the workpiece position. Therefore, it is possible to determine the position of the welding line with high accuracy for robotic welding.

[0064] [Point Group Data]

[0065] Figure 3 An example of point group data obtained by capturing images of workpiece 300 using a 3D sensor 200 is shown. Here, a diagram is shown as an example of workpiece 300 captured from above and viewed from an oblique angle. In this example, workpiece 300 is a cross-shaped workpiece comprising a central partition and four beam flanges extending along its four directions when viewed from above. As an example of workpiece 300, a steel frame joint workpiece is given.

[0066] The shape, structure, and dimensions of workpiece 300 are not particularly limited. For example, as examples of structures including partitions and beam flanges, there are T-shaped structures including one partition and three beam flanges, L-shaped structures including one partition and two beam flanges, and I-shaped structures. In addition, the presence or absence of a step at the connection position between the partition and the beam flange, and the offset at the connection position between the partition and the beam flange are given as structural elements.

[0067] Figure 4 This diagram illustrates the elements that constitute the measurement object of the workpiece 300 in the workpiece measurement method of this embodiment. Here, an example of an L-shaped workpiece including a partition 301 and two beam flanges 302, 303 is shown. Center point 301a represents the center of the partition 301. Center line 301b is a center line passing through the center point 301a of the partition 301 and along the x-axis. Center line 301c is a center line passing through the center point 301a of the partition 301 and along the y-axis. Center line 302a is the center line of the beam flange 302 along the x-axis, corresponding to the center position of the short side of the beam flange 302. Center line 303a is the center line of the beam flange 303 along the y-axis, corresponding to the center line of the short side of the beam flange 303. Assuming center line 302a is parallel to center line 301b, their difference is represented as offset OFF1. Assuming center line 303a is parallel to center line 301c, their difference is represented as offset OFF2. In the following description, the shorter side of the beam flange will be referred to as the width, and the longer side as the length.

[0068] In the following description, the point group data and the shape of the workpiece described above are used as examples. However, the invention is not limited to workpieces of such shapes, and other shapes of workpieces can be used as the object of this invention.

[0069] [Processing Flow]

[0070] The following describes the workpiece measurement process of this embodiment. Figure 5 This is a flowchart illustrating the overall process flow of the workpiece measurement process according to this embodiment. Each step is performed through... Figure 2 The information processing device 100 shown is implemented through the cooperation of its various parts. Here, for the sake of simplicity, the main processing unit will be referred to as the information processing device 100. Before starting this processing flow, the workpiece 300 is positioned at a location that can be captured by the three-dimensional sensor 200.

[0071] In step S501, the information processing device 100 acquires three-dimensional data, i.e., point group data, captured by the three-dimensional sensor 200. If the workpiece 300 is smaller than a specified size, the point group data can be acquired in a single shooting action. Conversely, if the workpiece 300 is larger than a specified size, multiple point group data can be acquired through multiple shooting actions and then combined. Furthermore, if the workpiece 300 has a specified shape, such as a steel frame joint, it is preferable to shoot from directly above the workpiece 300 to minimize blind spots. In this process, the information processing device 100 can adjust the shooting position and angle during shooting, or the user of the workpiece measurement system 1 can specify such a structure.

[0072] In step S502, the information processing device 100 preprocesses the point group data obtained in step S501. Preprocessing may include, for example, filtering, outlier removal, clustering, and coordinate transformation. It should be noted that the preprocessing in this step can be performed according to the structure of the point group data obtained in step S501, or it may be omitted.

[0073] Filtering processing can be performed as follows: using a known Voxel Grid filter, the point groups contained in the point group data are resampled at fixed intervals, thereby fixing the point group density per specified volume. Outlier removal processing can be performed to remove outliers that may reduce measurement accuracy. Outliers can be determined, for example, based on statistical information such as the mean and variance of adjacent point groups, or based on the number of adjacent point groups within a specified radius. Clustering processing can be performed as follows: dividing the point groups contained in the point group data into multiple groups based on distance, etc., and deleting groups whose number of member point groups is below a specified threshold, thereby removing point groups other than those representing the shape of the workpiece 300. Coordinate transformation processing transforms the coordinate system of the three-dimensional sensor 200 into a specified coordinate system based on the shooting position and shooting angle of the three-dimensional sensor 200. The specified coordinate system can be, for example, the coordinate system used in the touch sensing function, or... Figure 1 The diagram shows a coordinate system with the surface containing workpiece 300 as the xy plane and a defined origin and axes. The parameters required for coordinate system transformation can be derived in advance through prior calibration or other processes.

[0074] In step S503, the information processing device 100 uses the point group data processed in step S502 to perform probabilistic inference processing. Then it uses... Figures 6-8 This process is described in detail. As a result of this process, one or more enclosure boxes representing the general shape of workpiece 300 are obtained.

[0075] In step S504, the information processing device 100 performs optimization processing to correct the bounding box obtained in step S503. Then it uses... Figure 9 This process will be described in detail.

[0076] In step S505, the information processing device 100 outputs supplementary information for determining the position coordinates of the workpiece 300 in three-dimensional coordinates. The supplementary information includes, for example, the height information of each component constituting the workpiece 300. Figure 3 As shown, the upper surfaces of the partitions and beam flanges constituting workpiece 300 do not necessarily have to be horizontal; their heights can vary depending on their positions. Therefore, to more accurately determine the shape of workpiece 300, height information can be included as supplementary information. In addition, the angles formed by the surfaces of each part to the horizontal plane can also be included as supplementary information.

[0077] Figure 13 This is an explanatory diagram used to illustrate the calculation of the height parameter in the z-axis direction, which serves as supplementary information. In this example, it is related to... Figure 3 Similarly, an example of a cross-shaped workpiece 300, consisting of a partition and four beam flanges, is shown. Regarding the partition, considering the low measurement accuracy of the point group data at the periphery, the height of the partition is calculated using height information from region 1301, excluding the point group data located within a defined range from the boundary. Similarly, the beam flanges are also treated as having low measurement accuracy due to the low measurement accuracy of the point group data at the periphery. Furthermore, height information from locations close to the partition is used. Therefore, when calculating the height of each beam flange, the height information from the point group data in region 1302 is used.

[0078] In step S506, the information processing device 100 performs correction processing based on the information obtained through the touch sensing function. Regarding the actions in this step, they can also be performed after the information processing device 100 determines whether to perform a touch sensing action based on the touch sensing function and the correction processing based on the measurement results obtained through the current processing. Alternatively, the user of the workpiece measurement system 1 can specify whether to perform this step. Therefore, this step can also be omitted. Then, the processing flow ends.

[0079] (Probability Inference Processing)

[0080] Figure 6 This is a flowchart of the probabilistic deduction process in this embodiment, corresponding to... Figure 5 The process in step S503. First, using... Figure 7 and Figure 8 A general overview of the probabilistic inference process is provided.

[0081] Figure 7This shows point group data when the workpiece 300 is captured from directly above by a 3D sensor 200. That is, it is based on... Figure 3 The point group data shown has been transformed in the way it is projected onto a two-dimensional plane. In this example, workpiece 300 has a cross shape. In the probabilistic estimation process, probabilistic estimation is performed sequentially from multiple directions defined based on the shape of the workpiece 300, which is the object of measurement. For example, in the case that workpiece 300 is cross-shaped, actions are performed in four directions: right to left along the x-axis, left to right, top to bottom along the y-axis, and bottom to top. Figure 7 In this case, an example is shown along the x-axis from left to right. Additionally, a ROI (Region of Interest) of 701 is set, and probabilistic inference is performed while shifting the ROI 701. Figure 7 The diagram shows an example of how the width of the left-hand beam flange among the four beam flanges constituting workpiece 300 can be inferred from left to right along the x-axis.

[0082] Figure 8 It is used for explanation Figure 7 The diagram shown illustrates the probabilistic inference process. Figure 7 The coordinate graph of the example of the result of the probabilistic inference. Figure 8 In the middle, the horizontal axis indicates the direction of the first axis ( Figure 7 In the example, the detection results are shown on the x-axis direction, and the y-axis shows the second axis direction. Figure 7 In the example, the detection results are along the y-axis. Figure 7 In the measurement example, the horizontal axis corresponds to the length of the beam flange, and the vertical axis corresponds to the width of the beam flange. Additionally, on the horizontal axis, the position where the detection point group will begin during the movement of ROI701 is represented as "0". The values ​​for each axis are example and can vary depending on the dimensions of the workpiece 300 being measured. (The last sentence appears to be incomplete and possibly refers to a different measurement method.) Figure 7 By determining the point group data corresponding to the beam flange as shown, the width of the beam flange constituting workpiece 300, i.e., the general shape of workpiece 300, can be inferred. In this example, the width of the beam flange can be inferred as "200". Furthermore, during the continuous inference process, the width value increases sharply at the position of length "500". Therefore, the length of the beam flange can be inferred as "500".

[0083] As mentioned above, in Figure 7 In the example, probabilistic inference is performed by scanning the point group data from four directions. At this point, it can also be done as follows: Figure 8In this way, the direction of probabilistic inference is switched when the measured value changes drastically. Alternatively, probabilistic inference can be performed from all directions for all point group data. In the following description, the method of switching the direction of probabilistic inference when the measured value changes above a threshold will be explained.

[0084] Information about the bounding box obtained through probabilistic inference can be exemplified by items such as those shown below. It should be noted that, depending on the shape of the workpiece 300 being measured, the user may specify a portion of the information as shown below.

[0085] Shapes: rectangle, circle, cube, etc.

[0086] Dimensions: In the case of a rectangular shape, the length of the longer side, the length of the shorter side, the center coordinates, and the angles, etc.

[0087] Number: The number of bounding boxes for each shape

[0088] Constraints: Connection relationships and positional relationships between bounding boxes, etc.

[0089] return Figure 6 , to conduct with Figure 7 The flowchart corresponding to the processing is explained. As a premise of this processing flow, when performing the probabilistic deduction of workpiece 300 to obtain the bounding box information as described above, conditions corresponding to the shape of workpiece 300 are specified. For example, when workpiece 300 is Figure 3 In the case of a cross-shaped workpiece with a steel joint, the following conditions are given for probabilistic deduction.

[0090] Shape: Rectangle

[0091] Size: Varies depending on the component.

[0092] Quantity: 1 partition plate + 2-4 beam flanges

[0093] Constraints: The short side of the beam flange connects to one side of the partition plate; different beam flanges do not connect to each other.

[0094] Under the aforementioned inference conditions, a general shape estimation of workpiece 300 is performed. It should be noted that the inference conditions for the general shape estimation are predetermined based on the shape of workpiece 300. Furthermore, the inference conditions are not limited to those described above, and any arbitrary inference conditions can be specified. The conditions used for performing the aforementioned general shape estimation can also be predetermined based on the type of workpiece 300 that is the object of measurement.

[0095] In addition, in order to deduce the general shape of the joints of the components constituting the workpiece 300, it is necessary to start from above the workpiece 300, that is, along... Figure 1 Observe along the z-axis direction, etc. Therefore, when using Figure 3 In the case of the point group data shown, rotate the point group data to convert it from... Figure 7 The view shown is from above. This transformation can be achieved through... Figure 5 The preprocessing in step S502 is performed.

[0096] For point group data, according to Figure 1 The loop process of steps S601 to S611 is repeated at various specified angles around the z-axis in the xy-plane as shown. The specified angles and the starting angle are predetermined. For example, the specified angle can be set to 5 degrees, and the starting angle can be set to an angle (0 degrees) based on the x-axis direction in the xy-plane.

[0097] In step S602, the probabilistic inference processing unit 153 sets ROI 701 on the xy plane of the point group data. The ROI 701 set here is equivalent to the starting position of the probabilistic inference, and multiple ROIs can be set according to the number of scanning directions, i.e., the number of scanning directions of the point group data. In addition, the size of ROI 701 is not particularly limited. For example, a fixed size can be used, or it can be specified according to the image size of the point group data.

[0098] Next, for the point group data, the cyclic processing of steps S603 to S609 is repeated according to each specified direction on the xy plane. Here, the point group data is scanned in four scanning directions: the positive x-axis, the positive y-axis, the negative x-axis, and the negative y-axis. The scanning direction and the number of scans can be specified based on the above-mentioned inference conditions.

[0099] In step S604, the probabilistic inference processing unit 153 calculates the width of the point group in the set ROI 701.

[0100] In step S605, the probabilistic estimation processing unit 153 determines whether the difference between the width of the point group calculated in step S604 and the width calculated at the previous position of ROI 701, i.e., the change amount, is above a threshold. This threshold is predetermined. It should be noted that the change rate can also be used instead of the change amount for determination. If the change amount is above the threshold ("Yes" in step S605), the probabilistic estimation processing unit 153 proceeds to step S607. On the other hand, if the change amount is less than the threshold ("No" in step S605), the probabilistic estimation processing unit 153 proceeds to step S606.

[0101] In step S606, the profiling inference processing unit 153 moves the ROI 701 parallel along the scanning direction. The amount of movement of the ROI 701 is predetermined based on the size of the ROI 701, etc. Then, the processing of the profiling inference processing unit 153 returns to step S604, and the processing is repeated.

[0102] In step S607, the probabilistic inference processing unit 153 sets the value immediately preceding the position where the change amount is above the threshold as the width of the beam flange, and sets the length from the position of the detected point group to the position where the change amount is above the threshold as the length of the beam flange.

[0103] In step S608, if the length of the beam flange set in step S607 is below a threshold, the probabilistic estimation processing unit 153 sets it to indicate that there is no beam flange in that direction. That is, the number of beam flanges is set.

[0104] In step S609, the probabilities estimation processing unit 153 determines whether scanning in all scanning directions is complete. If scanning in all scanning directions is not complete, it switches to the unprocessed scanning direction and repeats the processing after step S604. On the other hand, if scanning in all scanning directions is complete, the processing of the probabilities estimation processing unit 153 proceeds to step S610.

[0105] In step S610, the probabilistic estimation processing unit 153 calculates the angle between the centerline of each beam flange and the x-axis or y-axis on the xy plane.

[0106] In step S611, the probabilistic estimation processing unit 153 determines whether the calculation of all rotation angles is complete. If the calculation of all rotation angles is not complete, the rotation is changed to the next angle, and the processing after step S602 is repeated. On the other hand, if the calculation of all rotation angles is complete, the processing of the probabilistic estimation processing unit 153 proceeds to step S612.

[0107] In step S612, the profiling and processing unit 153 sets the rotational position of the workpiece 300 at which the average angle between the centerline of each beam flange constituting the workpiece 300 and the x-axis or y-axis is minimized. That is, in the xy plane, the rotation angle of the point group data around the z-axis is determined to make the x-axis and y-axis parallel to each beam flange.

[0108] In step S613, the profiling processing unit 153 calculates the dimensions of the partitions constituting the workpiece 300 based on the overall width and length of the point group and the calculated lengths of each beam flange. The dimensions of the partitions can be calculated based on the aforementioned inference conditions. Thus, one or more bounding boxes representing the overall profiling of the workpiece 300 can be generated. For example, in a workpiece having… Figure 7In the case of workpiece 300 with the given shape, a total of five bounding boxes can be generated, each corresponding to one partition and four beam flanges. In other words, the general shape of each component constituting workpiece 300 is determined by the parameters of the bounding boxes. Then, this processing flow ends.

[0109] (Bounding box optimization)

[0110] Figure 9 This is a flowchart of the bounding box optimization process in this embodiment, corresponding to... Figure 5 The process of step S504.

[0111] In step S901, the bounding box optimization unit 154 will pass through Figure 5 The parameters of each bounding box obtained through the probabilistic inference process in step S503 are set as initial values. That is, the parameters obtained through the probabilistic inference process are used as a reference, and the measurement accuracy is improved through optimization processing.

[0112] In step S902, the bounding box optimization unit 154 uses an evaluation function to calculate an evaluation value. In this embodiment, the following equation (1) is used as the evaluation function, which represents a weighted linear sum obtained based on the number of points in the point group and the density of the point group.

[0113] F(x)=WNN(x)+WDD(x)…(1)

[0114] WN, WD: Weighting coefficients

[0115] x: Optimization parameters

[0116] N(x): The number of points within the bounding box

[0117] D(x): Point group density within the bounding box

[0118] Optimize parameter x, for example, by using the position, angle, width, and length of the bounding box. More specifically, in Figure 4 In the case of the workpiece 300 with the structure shown, the angle of the workpiece 300 around the z-axis, the center coordinates of the partition, the width, length, and offset of the beam flange can be used as optimization parameters.

[0119] In step S903, the bounding box optimization unit 154 calculates the change in the evaluation value based on the initial value set in step S901 and the evaluation value calculated in step S902. In this embodiment, the change in the evaluation value is calculated while changing each optimization parameter at predetermined intervals starting from the initial value.

[0120] In step S904, the bounding box optimization unit 154 updates the parameters based on the gradient vector calculated in step S903. Here, for example, a known method such as the steepest descent method can be used to update the parameters.

[0121] In step S905, as a result of updating the parameters, the bounding box optimization unit 154 determines whether the change has converged within a specified range or whether the number of updates has reached a specified threshold. If the convergence is within the specified range or the number of updates reaches the specified threshold ("Yes" in step S905), the current processing flow ends. Otherwise ("No" in step S905), the processing of the bounding box optimization unit 154 returns to step S902 and the processing is repeated.

[0122] [Explanation of Processing Results]

[0123] Figures 10-12 This is a diagram showing the measurement results obtained by the workpiece measurement method of this embodiment. Here, as a conventional method compared with the workpiece measurement method of this embodiment, a method based on the well-known OBB (Oriented Bounding Box) will be described as an example.

[0124] Figure 10 An example of a bounding box is shown as a measurement result using point group data 1000 of a certain workpiece. Fixed noise is contained around the point group data 1000. Bounding box 1001 shows the measurement result obtained by a conventional method, and bounding box 1002 shows the measurement result obtained by the workpiece measurement method of this embodiment. In conventional methods, as a result of noise, a range larger than that representing the actual workpiece is shown. On the other hand, in the workpiece measurement method of this embodiment, a bounding box 1002 that is closer to the shape of the actual workpiece is obtained.

[0125] Figure 11 An example of a bounding box representing the measurement result using point group data 1100 of a certain workpiece is shown. In the point group data 1100, the workpiece was not correctly captured by the 3D camera, resulting in a defect 1101 in a portion. Bounding box 1102 represents the measurement result obtained by conventional methods, and bounding box 1103 represents the measurement result obtained by the workpiece measurement method of this embodiment. In conventional methods, as a result of the defect 1101, the principal axis cannot be accurately determined, resulting in errors in angles and dimensions for the point group representing the actual workpiece. On the other hand, in the workpiece measurement method of this embodiment, the influence of the defect 1101 is suppressed, resulting in a bounding box 1103 that more closely approximates the angles and dimensions of the actual workpiece.

[0126] Figure 12An example of a bounding box representing the measurement result using point group data 1200 for a certain workpiece is shown. In point group data 1200, accessories are provided at two locations on the workpiece, including point groups 1201 corresponding to these accessories. Bounding box 1202 represents the measurement result obtained by a conventional method, and bounding box 1203 represents the measurement result obtained by the workpiece measurement method of this embodiment. In conventional methods, as a result of the influence of point group 1201, a range larger than that representing the actual workpiece is shown. On the other hand, in the workpiece measurement method of this embodiment, point groups 1201 smaller than a specified size are removed, suppressing their influence, resulting in a bounding box 1203 that more closely approximates the shape of the actual workpiece.

[0127] According to this embodiment, the shape and position of a workpiece can be determined without prior preparation of 3D CAD data. Furthermore, compared to existing methods, the shape of the workpiece can be determined with higher precision.

[0128] <Other Implementation Methods>

[0129] The workpiece measurement method described in the above embodiments can be applied to welding systems including welding robots. Thus, for example, welding lines can be automatically extracted from the workpiece based on information from the bounding box.

[0130] Furthermore, while the above embodiment illustrates an example of determining the shape based on a viewpoint observed from above the workpiece, a structure capable of measuring the shape from multiple directions is also possible. This allows for the suppression of the effects of blind spots and other factors, resulting in more precise measurements.

[0131] This embodiment can also be implemented by the following process: using a network or storage medium to supply a program or application for implementing the functions of one or more embodiments described above to a system or device, and one or more processors in the computer of the system or device reads and executes the program.

[0132] Alternatively, this embodiment can also be implemented by a circuit that performs more than one function. It should be noted that examples of circuits that perform more than one function include ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays).

[0133] As stated above, the following matters are disclosed in this specification.

[0134] (1) A workpiece measurement method, which measures the shape and position of a workpiece composed of multiple components, wherein,

[0135] The workpiece measurement method has the following characteristics:

[0136] The acquisition step involves acquiring three-dimensional point group data of the workpiece.

[0137] A probabilistic inference step, in which one or more bounding boxes representing the probabilistic shape corresponding to the plurality of components are inferred using conditions specified corresponding to the shape of the workpiece being measured and the point group data; and

[0138] An optimization step is performed in which the shape inference step inferred one or more bounding boxes are optimized by adjusting the parameters using an evaluation function, thereby determining the shape of the plurality of components.

[0139] Based on this structure, the shape and position of a workpiece can be determined without prior preparation of 3D CAD data.

[0140] (2) According to the workpiece measurement method described in (1), wherein the conditions include any one of the shape, size, number and constraints of the bounding box corresponding to the workpiece.

[0141] Based on this structure, the bounding box of the workpiece can be inferred based on any one of the shape, size, number, and constraints that are conditions corresponding to the workpiece.

[0142] (3) The workpiece measurement method described in (1) or (2), wherein the evaluation function is a weighted linear sum obtained based on the number of points and the density of the point group contained in the bounding box.

[0143] According to this structure, the bounding box can be optimized based on a weighted linear sum obtained from the number of points and the density of the point group within the bounding box.

[0144] (4) The workpiece measurement method described in any one of (1) to (3), wherein, in the optimization step, at least one of the parameters of the bounding box, the size, the position, and the angle is optimized.

[0145] Based on this structure, it is possible to optimize at least one of the dimensions, position, and angle of the bounding box.

[0146] (5) The workpiece measurement method described in any one of (1) to (4), wherein, in the probabilistic inference step, the bounding box is inferred by scanning the point group data from multiple directions.

[0147] Based on this structure, the approximate shape of the workpiece can be inferred with greater accuracy.

[0148] (6) The workpiece measurement method described in any one of (1) to (5), wherein, in the probabilities inference step, the point group data is projected onto a two-dimensional plane to infer the probabilities.

[0149] Based on this structure, it is possible to infer with high accuracy the shape observed from the desired direction of projecting point group data onto a two-dimensional plane.

[0150] (7) According to the workpiece measurement method described in (6), there is also a derivation step in which the axial positions of the plurality of components orthogonal to the two-dimensional plane are derived.

[0151] Based on this structure, information about the height direction of each component constituting the workpiece relative to the two-dimensional plane can be further derived.

[0152] (8) The workpiece measurement method described in any one of (1) to (7), wherein the bounding box is configured to include a straight line or a curve.

[0153] Based on this structure, the shape of a workpiece can be determined using a bounding box of any shape.

[0154] (9) The workpiece measurement method described in any one of (1) to (8), wherein the bounding box is represented in two-dimensional or three-dimensional form.

[0155] Based on this structure, the shape of a workpiece can be determined using two-dimensional or three-dimensional bounding boxes.

[0156] (10) The workpiece measurement method according to any one of (1) to (9), wherein a correction step is further provided, in which the measurement results of touch sensing for the workpiece are used to correct the one or more bounding boxes optimized in the optimization step.

[0157] Based on this structure, the measurement accuracy can be further improved by using the results of touch sensing.

[0158] (11) A workpiece measurement system for measuring the shape and position of a workpiece composed of multiple components, wherein,

[0159] The workpiece measurement system has the following features:

[0160] The acquisition unit acquires the three-dimensional point group data of the workpiece;

[0161] A prototyping unit, using conditions specified corresponding to the shape of the workpiece being measured and the point group data, infers one or more bounding boxes representing the prototyping of the plurality of components; and

[0162] The optimization unit optimizes the one or more bounding boxes inferred by the probabilistic inference unit by adjusting the parameters using the evaluation function, thereby determining the shape of the plurality of components.

[0163] Based on this structure, the shape and position of a workpiece can be determined without prior preparation of 3D CAD data.

[0164] (12) A program in which,

[0165] The program is used to cause the computer to perform the following steps:

[0166] The acquisition step involves acquiring three-dimensional point group data of a workpiece composed of multiple components.

[0167] A probabilistic inference step, in which one or more bounding boxes representing the probabilistic shape corresponding to the plurality of components are inferred using conditions specified corresponding to the shape of the workpiece being measured and the point group data; and

[0168] An optimization step is performed in which the shape inference step inferred one or more bounding boxes are optimized by adjusting the parameters using an evaluation function, thereby determining the shape of the plurality of components.

[0169] Based on this structure, the shape and position of a workpiece can be determined without prior preparation of 3D CAD data.

Claims

1. A workpiece measurement method, which measures the shape and position of a workpiece composed of multiple components, wherein, The workpiece measurement method has the following characteristics: The acquisition step involves acquiring three-dimensional point group data of the workpiece. A probabilistic inference step, in which one or more bounding boxes representing the probabilistic shape corresponding to the plurality of components are inferred using conditions specified corresponding to the shape of the workpiece being measured and the point group data. An optimization step is performed in which the shape inference step infers one or more bounding boxes by adjusting parameters using an evaluation function, thereby determining the shape of the plurality of components. as well as A calibration step, in which the measurement results of touch sensing of the workpiece are used to calibrate the one or more bounding boxes optimized in the optimization step.

2. The workpiece measurement method according to claim 1, wherein, The conditions include any one of the shape, size, number, and constraints of the bounding box corresponding to the workpiece.

3. The workpiece measurement method according to claim 1 or 2, wherein, The evaluation function is a weighted linear sum based on the number of points and the density of the point group contained in the bounding box.

4. The workpiece measurement method according to claim 1 or 2, wherein, In the optimization step, at least one of the parameters of the bounding box, including its size, position, and angle, is optimized.

5. The workpiece measurement method according to claim 1 or 2, wherein, In the probabilistic inference step, the bounding box is inferred by scanning the point group data from multiple directions.

6. The workpiece measurement method according to claim 1 or 2, wherein, In the probabilistic inference step, the point group data is projected onto a two-dimensional plane to infer the probabilistic shape.

7. The workpiece measurement method according to claim 6, wherein, The workpiece measurement method further includes an export step, in which the axial positions corresponding to the plurality of components and orthogonal to the two-dimensional plane are exported.

8. The workpiece measurement method according to claim 1 or 2, wherein, The bounding box is configured to include straight lines or curves.

9. The workpiece measurement method according to claim 1 or 2, wherein, The bounding box is represented in two-dimensional or three-dimensional form.

10. A workpiece measurement system for measuring the shape and position of a workpiece composed of multiple components, wherein, The workpiece measurement system has the following features: The acquisition unit acquires the three-dimensional point group data of the workpiece; A prototyping unit uses conditions specified in relation to the shape of the workpiece being measured and the point group data to infer one or more bounding boxes representing the prototyping of the plurality of components. The optimization unit optimizes the one or more bounding boxes inferred by the probabilistic inference unit by adjusting the parameters using the evaluation function, thereby determining the shape of the plurality of components; as well as The correction unit uses the measurement results of touch sensing of the workpiece to correct the one or more bounding boxes optimized in the optimization unit.

11. A computer program product comprising a computer program, wherein, The computer program is used to cause the computer to perform the following steps: The acquisition step involves acquiring three-dimensional point group data of a workpiece composed of multiple components. A probabilistic inference step, in which one or more bounding boxes representing the probabilistic shape corresponding to the plurality of components are inferred using conditions specified corresponding to the shape of the workpiece being measured and the point group data. An optimization step is performed in which the shape inference step infers one or more bounding boxes by adjusting parameters using an evaluation function, thereby determining the shape of the plurality of components. as well as A calibration step, in which the measurement results of touch sensing of the workpiece are used to calibrate the one or more bounding boxes optimized in the optimization step.

Images