Program, data processing device, and data processing method

Abstract

To reduce variation in the positions of teaching points.SOLUTION: A presentation process is executed to present candidate teaching points to a user. A teaching point determination process is executed to determine a new target teaching point according to a user instruction. The teaching point determination process includes a process of determining the new target teaching point as the presented candidate teaching point when the user instruction is an adoption instruction. A next candidate determination process is executed to determine a candidate teaching point to be presented next on the basis of one or more target teaching points including the new target teaching point. A plurality of target teaching points are determined by executing a determination process. The determination process includes a process to sequentially add one new target teaching point to the plurality of target teaching points by repeating the presentation process, the teaching point determination process, and the next candidate determination process.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to techniques for determining a plurality of teaching points that form a path for machining an object with a tool. [Background technology]

[0002] A technology is used that performs direct teach control in response to the application of an external force to the robot's hand. Patent Document 1 proposes a method in which a worker touches a work tool to the robot to perform direct teach, rather than directly touching the robot. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-297853 Summary of the Invention [Problem to be solved by the invention]

[0004] The machining path is formed by a plurality of teaching points determined by teaching. The positions of the teaching points may differ depending on the proficiency of the operator who performs the teaching. That is, the positions of the teaching points may vary due to variations in the operator's proficiency. The variations in the positions of the teaching points may cause variations in the machining path. The variations in the machining path may cause variations in the machining quality.

[0005] This specification discloses a technique for reducing the variation in the positions of teaching points. [Means for solving the problem]

[0006] The techniques disclosed in this specification can be implemented in the following application examples.

[0007] [Application Example 1] A program that causes a computer to realize the following functions: a function of executing a presentation process that presents to a user candidate teaching points that are candidates for target teaching points that form a path for machining an object by a tool; a function of executing a teaching point determination process that determines a new target teaching point in accordance with an instruction from the user, wherein the teaching point determination process includes a process of determining the new target teaching point as the presented candidate teaching point when the instruction from the user is an adoption instruction indicating the adoption of the presented candidate teaching point; a function of executing a next candidate determination process that determines a candidate teaching point to be presented next based on one or more target teaching points including the new target teaching point; and a function of determining multiple target teaching points by executing a determination process, wherein the determination process includes a process of adding new target teaching points one by one to the multiple target teaching points by repeating the presentation process, the teaching point determination process, and the next candidate determination process.

[0008] According to this configuration, a plurality of target teaching points are determined by executing a determination process, and the determination process includes a process of adding new target teaching points one by one to the plurality of target teaching points by repeating a presentation process, a teaching point determination process, and a next candidate determination process, and the teaching point determination process includes a process of determining the new target teaching point as the presented candidate teaching point when the user's instruction is an adoption instruction, thereby reducing variation in the positions of the teaching points.

[0009] The technology disclosed in this specification can be realized in various forms, such as a data processing method and a data processing device, a computer program for realizing the functions of the method or device, a recording medium (e.g., a non-temporary recording medium) on which the computer program is recorded, and the like. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is an explanatory diagram illustrating a data processing device according to an embodiment; [Figure 2] FIG. 9 is a perspective view illustrating an example of a processing device 900. [Figure 3]10 is a flowchart illustrating an example of a teaching process. [Figure 4] 10 is a flowchart illustrating an example of a teaching process. [Figure 5] 10A to 10G are diagrams showing examples of information processed by the teaching process. [Figure 6] 10 is a flowchart illustrating an example of a process for determining a candidate teaching point. [Figure 7] (A) is a diagram showing an example of a specific range VRs. (B) is a diagram showing an example of a candidate teaching point cPt. (C) is a diagram showing an example of a plurality of representative points Pr and a specific range VRs. (D) is a diagram showing an example of a nearby range VRn. (E) is a diagram showing an example of a candidate teaching point cPt and a corrected candidate teaching point cPtm. (F) is a diagram showing an example of a candidate approach point cPtS. (G) is a diagram showing an example of a candidate escape point cPtE. [Figure 8] 10 is a part of a flowchart illustrating another embodiment of a teaching process. [Figure 9] 10 is a diagram showing an example of an image displayed on a display unit 240. FIG. [Figure 10] FIG. 10 is a diagram showing an example of candidate teaching points cPt. [Figure 11] 1A is a diagram showing an example of the arrangement of a plurality of voxels VX. FIG. 1B is a diagram showing an example of a plurality of representative points Pr. DETAILED DESCRIPTION OF THE INVENTION

[0011] A. First Example: A1.Device configuration: 1 is an explanatory diagram showing a data processing device according to an embodiment. The data processing device 200 is, for example, a personal computer. The data processing device 200 executes a process for determining a plurality of teaching points that form a path for machining an object using a tool.

[0012] The data processing device 200 includes a processor 210, a storage device 215, a display unit 240, an operation unit 250, and a communication interface 270. These elements are connected to each other via a bus. The storage device 215 includes a volatile storage device 220 and a non-volatile storage device 230.

[0013] The processor 210 is a device configured to perform data processing, and is, for example, a central processing unit (CPU) or a system on a chip (SoC). The volatile storage device 220 is, for example, a dynamic random access memory (DRAM), and the nonvolatile storage device 230 is, for example, a flash memory. The nonvolatile storage device 230 stores data of a program 231. Details of the data stored in the nonvolatile storage device 230 will be described later.

[0014] The display unit 240 is a device configured to display images, such as a liquid crystal display or an organic EL display. The operation unit 250 is a device configured to receive operations by a user, such as a button, a lever, or a touch panel overlaid on the display unit 240. The display unit 240 and the operation unit 250 may form a so-called touch screen. The user can input various requests and instructions to the data processing device 200 by operating the operation unit 250. The display unit 240 may display operation elements (e.g., buttons, sliders, etc.), and the displayed elements may be operated through operation of the operation unit 250.

[0015] The communication interface 270 is an interface for communicating with other devices (for example, it includes one or more of a USB interface, a wired LAN interface, an IEEE802.11 wireless interface, and an industrial camera interface (for example, CameraLink, CoaXPress, etc.)). In this embodiment, the processing device 900, the camera 700, and the controller 600 are connected to the communication interface 270. The controller 600 is a device configured to receive operations by a user. In this embodiment, the controller 600 has a lever and a plurality of buttons. The user can input various requests and instructions to the data processing device 200 by operating the controller 600. The controller 600 may be used in place of the operation unit 250.

[0016] FIG. 2 is a perspective view illustrating an example of a processing apparatus 900. The processing apparatus 900 is a robot that processes a workpiece using a tool. The processing apparatus 900 includes a support device 930, a tool unit 990u having a tool 990, and a control device 980. The support device 930 includes a first support device 910 configured to support the tool unit 990u and a second support device 920 configured to support the workpiece 800. The figure illustrates mutually perpendicular X, Y, and Z directions. A relative position with respect to the processing apparatus 900 is represented by three-dimensional coordinates of X, Y, and Z (referred to as a robot coordinate system). The robot coordinate system is a Cartesian coordinate system fixed in advance with respect to the processing apparatus 900. The origin of the robot coordinate system may be located at various positions. Hereinafter, the X direction will also be referred to as the +X direction, and the direction opposite to the X direction will also be referred to as the -X direction. The same applies to the Y and Z directions.

[0017] The first support device 910 includes three arms 910x, 910y, and 910z. The first arm 910x extends in the X direction and is fixed to a frame (not shown). The first arm 910x supports a second arm 910y so as to be slidable in a direction parallel to the X direction. The second arm 910y extends in the Y direction and is slidable in a direction parallel to the Y direction relative to the first arm 910x. The second arm 910y supports a third arm 910z so as to be slidable in a direction parallel to the Z direction. The third arm 910z extends in the Z direction. A tool unit 990u is attached to the end of the third arm 910z on the -Z direction side. Although not shown, the first support device 910 has sensors (e.g., encoders) that detect the slide positions in the X, Y, and Z directions.

[0018] Each of the arms 910x, 910y, and 910z may have a variety of configurations, including the above-described sliding device. For example, the first arm 910x may have a rail that supports the second arm 910y so that the second arm 910y can slide in a direction parallel to the X direction, multiple pulleys, a belt that is wound around the multiple pulleys and has a portion fixed to the second arm 910y, and an electric motor that rotates the pulley. The arms 910y and 910z may also have a similar configuration.

[0019] The tool 990 may be any of various tools for processing the workpiece 800. In this embodiment, the workpiece 800 is formed by machining, such as cutting or drilling, or by casting (e.g., a machine part). Such a workpiece 800 may have a burr. The tool 990 forms an outer surface with a blade for deburring the workpiece 800 (such a tool 990 is also called a spindle). The tool 990 extends along a central axis 990ax parallel to the Z direction. Although not shown, the tool unit 990u has an electric motor that rotates the tool 990 around the central axis 990ax. The rotating tool 990 comes into contact with the burr on the workpiece 800, thereby scraping the burr off. The +Z direction indicates the direction in which the tool 990 moves away from the workpiece 800.

[0020] The second support device 920 includes two support walls 920s and a table 920t disposed between the two support walls 920s. The two support walls 920s support the table 920t so that it can rotate about a rotation axis Da. Although not shown, the second support device 920 includes an electric motor that rotates the table 920t and a sensor (e.g., an encoder) that detects the rotation position (i.e., angle). In this embodiment, the rotation axis Da is parallel to the X direction. The workpiece 800 is attached to the table 920t by a fixture (not shown).

[0021] The control device 980 is an electric circuit configured to control the first support device 910, the second support device 920, and the tool 990. The control device 980 can control the support devices 910, 920, and the tool 990 according to instructions from an external device (e.g., the data processing device 200). The control device 980 may be configured using a computer or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC)).

[0022] The control device 980 can change the relative position of the tool 990 with respect to the workpiece 800 by controlling the first support device 910. Specifically, the position expressed in three dimensions changes. The control device 980 can change the relative attitude (here, position and orientation) of the tool 990 with respect to the workpiece 800 by controlling the second support device 920. Specifically, the position expressed in three dimensions and the orientation expressed in three dimensions change.

[0023] In this way, the processing device 900 can position the tool 990 in various positions and orientations relative to the workpiece 800. For example, by combining movement by the first support device 910 and rotation by the second support device 920, the control device 980 can change the orientation of the tool 990 relative to the workpiece 800 without changing the position of the tip 990t of the tool 990 relative to the workpiece 800. Specifically, the process is as follows. Assume that the tip 990t of the tool 990 is located at a specific position relative to the workpiece 800. The second support device 920 rotates the workpiece 800 together with the table 920t around the rotation axis Da. The specific position relative to the workpiece 800 rotates together with the workpiece 800 around the rotation axis Da. Due to this rotation, the orientation of the tool 990 relative to the workpiece 800 changes. Furthermore, the position of the tool 990 relative to the workpiece 800 also changes. The first support device 910 moves the tool 990 to the specific position to which it has been moved. As a result, the tool 990 changes its orientation relative to the workpiece 800 without changing its position.

[0024] In response to a request from an external device (e.g., data processing device 200), the control device 980 can provide the external device with data representing the current slide positions in the X, Y, and Z directions by the support device 910 and the rotational position by the second support device 920.

[0025] Burrs may be formed on the workpiece 800. To remove the burrs, the processing device 900 moves the tool 990 along the burrs. Such a processing path of the tool 990 (i.e., the movement path of the tool 990) is formed by a plurality of teaching points arranged along the path. The data processing device 200 determines the plurality of teaching points by a teaching process described below.

[0026] The camera 700 includes a sensor that acquires data of a two-dimensional image and a sensor that measures the three-dimensional coordinates of each of a plurality of points on the two-dimensional image. The position and orientation of the camera 700 are adjusted in advance so that it can read an object (e.g., workpiece 800) on the table 920t. Although not shown in the figure, the reading by the camera 700 is performed in a state where a light source is illuminating the object on the table 920t.

[0027] The camera 700 acquires a two-dimensional image including an image of the workpiece 800 and the three-dimensional coordinates of each of a plurality of points on the two-dimensional image (including a plurality of points on the image of the outer surface of the workpiece 800). In this embodiment, the data of the two-dimensional image is bitmap data that represents color values ​​using three color components: red R, green G, and blue B. Although not shown, the two-dimensional image is represented by the color values ​​of each of a plurality of pixels arranged in a matrix (the color values ​​indicate the gradation values ​​of each of red R, green G, and blue B (for example, values ​​greater than or equal to zero and less than or equal to 255)). The three-dimensional coordinates are represented in a Cartesian coordinate system (also called a camera coordinate system) that is fixed in advance relative to the camera 700.

[0028] The camera 700 may be, for example, a camera called a depth camera. Alternatively, the camera 700 may include a camera for generating (i.e., photographing) two-dimensional images and a three-dimensional scanner (also called a distance sensor). The three-dimensional coordinates may be acquired by various methods. The three-dimensional coordinates may be acquired by, for example, stereoscopic vision, structured light patterns, or time-of-flight.

[0029] The multiple points captured by the camera 700 are also referred to as a point cloud. The data processing device 200 determines multiple teaching points that form a machining path suitable for the workpiece 800 by using the coordinates of each point of the point cloud in the teaching process described below. In this embodiment, the teaching points represent the position and orientation of the tool 990 relative to the workpiece 800. Hereinafter, the position of the tool 990 represented by the teaching points will also be referred to as the teaching position. The orientation of the tool 990 represented by the teaching points will also be referred to as the teaching orientation. The orientation of the tool 990 may be, for example, a direction from the tip 990t of the tool 990 to the base of the tool 990, parallel to the central axis 990ax of the tool 990. In the example of FIG. 2, the orientation of the tool 990 is the same as the +Z direction.

[0030] A2. Teaching process: 3 and 4 are flowcharts showing an example of the teaching process. FIG. 4 is a continuation of FIG. 3. In S110, a user (e.g., an operator) colors the portion of the outer surface of the workpiece 800 to be machined with a specific color. In this embodiment, a portion that is likely to form burrs is used as the portion to be machined. Any coloring method may be used. The user may use, for example, a marker pen or colored tape to color the portion to be machined.

[0031] 5(A)-5(G) are diagrams showing examples of information processed by the teaching process. FIG. 5(A) shows an example of a workpiece 800 and a target area 800A. In the example of FIG. 5(A), the outer surface 800o of the workpiece 800 forms a vertex V1 and three sides E1, E2, and E3 extending from the vertex V1 in different directions. The target area 800A in the figure is an area of ​​the outer surface 800o of the workpiece 800 that is colored with a specific color. In the example of FIG. 5(A), the portions of the outer surface 800o that form the sides E1, E2, and E3 are colored.

[0032] The color of the outer surface 800o of the workpiece 800 may be various colors. The specific color may be various colors different from the color of the outer surface 800o. For example, if the color of the outer surface 800o is silver, the specific color may be red.

[0033] After S110 (FIG. 3), the processor 210 of the data processing device 200 (FIG. 1) starts the process for teaching according to the program 231 in response to a start instruction for teaching input to the data processing device 200. The start instruction may be input by any method. In this embodiment, the user inputs the start instruction by operating the operation unit 250. The user may input the start instruction to the data processing device 200 through a terminal device (e.g., a tablet computer, etc.) (not shown) that can communicate with the data processing device 200.

[0034] In S120, the processor 210 supplies a reading instruction to the camera 700. In accordance with the reading instruction, the camera 700 reads the workpiece 800. As a result, the camera 700 generates data of a two-dimensional image including an image of the workpiece 800, and data of the three-dimensional coordinates of each of a plurality of points on the two-dimensional image (including a plurality of points on the image of the outer surface 800o of the workpiece 800).

[0035] In S130, the processor 210 acquires two-dimensional image data and three-dimensional coordinate data of each of a plurality of points from the camera 700. The three-dimensional coordinates are expressed in the camera coordinate system. The processor 210 uses all of this data as point cloud data.

[0036] In S140, the processor 210 extracts multiple points of a specific color from the point cloud. FIG. 5(B) shows an example of a point cloud of a specific color. The figure shows a portion of an example of a two-dimensional image IM acquired from the camera 700. The two-dimensional image IM includes an image of the workpiece 800. The multiple open circles in the figure indicate multiple points P included in the point cloud. The figure shows multiple points P located within the image of the target area 800A on the two-dimensional image IM. The three-dimensional coordinates associated with these points P indicate the positions of corresponding portions on the target area 800A of the outer surface 800o of the workpiece 800. Although not shown, the two-dimensional image IM may also represent a portion of the second support device 920 (e.g., a portion of the table 920t). The point cloud includes multiple points located in other portions of the workpiece 800 and multiple points located outside the workpiece 800.

[0037] Various methods may be used to extract multiple points of a specific color. In this embodiment, the processor 210 extracts a region from the two-dimensional image IM that has color values ​​within a specific color range. The specific color range is determined experimentally in advance so that the color values ​​of pixels representing portions colored in the specific color are within the specific color range, and the color values ​​of pixels representing colors different from the specific color are outside the specific color range. The processor 210 extracts multiple points P within the extracted region from the two-dimensional image IM. The extracted points P represent points on the target region 800A of the outer surface 800o of the workpiece 800. Note that the color values ​​of pixels representing portions colored in the specific color in the two-dimensional image IM may vary due to various causes, such as variations in the light intensity of a light source (not shown). The specific color range is determined so that multiple points of the specific color can be extracted even when the color values ​​vary. By using this specific color range, the processor 210 can extract multiple points P within the target region 800A.

[0038] The specific color range may be defined by the range of gradation values ​​of each of red (R), green (G), and blue (B). Alternatively, the specific color range may be defined by the range of other color components (e.g., the range of hue and the range of saturation). When luminance is omitted and one or both of hue and saturation are used, the possibility of extraction errors due to variations in the brightness of the light source is reduced. The processor 210 may calculate the hue and saturation from RGB according to a predetermined correspondence relationship (e.g., the correspondence relationship between the RGB color space and the HSV color space).

[0039] In S150 (FIG. 3), the processor 210 removes outliers from the point cloud of the specific color. This reduces the proportion of noise points included in the point cloud of the specific color. Various methods for removing outliers may be used. In this embodiment, the processor 210 removes outliers according to a method called radial outlier removal. Specifically, the processor 210 removes the point of interest P from the point cloud of the specific color if the total number of points P present within a three-dimensional sphere of a predetermined radius centered on the point of interest P is less than a number threshold (the radius and number threshold are determined in advance experimentally so that noise points are removed without removing appropriate points). To remove the outliers, for example, the function "remove_radius_outlier" included in a library called Open3D may be used. Alternatively, other methods may be used. For example, a method called statistical outlier removal may be used. To remove outliers using this method, for example, the function "remove_statistical_outlier" of Open3D may be used.

[0040] In S160, the processor 210 downsamples the point cloud of the specific color to obtain a sampled point cloud. Various downsampling methods may be used. In this embodiment, the processor 210 performs downsampling according to a so-called voxel grid method. A voxel is a three-dimensional cube. The processor 210 divides the three-dimensional space in which the point P exists into multiple voxels and obtains a representative point for each voxel. The processor 210 obtains multiple representative points of multiple voxels as a sampled point cloud.

[0041] S162 and S164 in the box S160 in FIG. 3 represent an example of steps of the voxel grid method. In S162, the processor 210 determines the size of a voxel. In this embodiment, the length of a side of a cube is determined as the size of the voxel (for example, in millimeters). The size of the voxel is determined so that multiple voxels (i.e., multiple representative points) arranged at a density determined by the size of the voxel can adequately represent the movement path of the tool 990. The method of determining the size of the voxel may be various. For example, the processor 210 may determine the length of the side to a predetermined value. Alternatively, the processor 210 may determine the length of the side in accordance with an instruction from a user.

[0042] In S164, the processor 210 arranges a plurality of voxels in the three-dimensional space where the point P exists. FIG. 5(C) shows an example of a plurality of voxels. The figure shows a workpiece 800 and a plurality of voxels VX. The processor 210 arranges the plurality of voxels VX in a lattice pattern. The figure shows a plurality of voxels VX that include a portion of the target region 800A, among the plurality of voxels VX.

[0043] In S164 (FIG. 3), the processor 210 further calculates a representative point Pr for each of the multiple voxels VX. FIG. 5(D) is an explanatory diagram of the representative point Pr. The diagram shows one voxel VX and multiple points P included in the voxel VX. The position of the representative point Pr may be various positions that represent the multiple positions of the multiple points P included in the voxel VX. In this embodiment, the processor 210 determines the position of the representative point Pr to be the position of the center of gravity of the multiple positions of the multiple points P. Alternatively, the position of the center of gravity of a three-dimensional convex hull including the multiple points P may be used. Note that if the total number of points P included in a voxel VX is zero, calculation of the representative point Pr for that voxel VX is omitted.

[0044] FIG. 5(E) shows an example of a plurality of representative points Pr. The figure shows a workpiece 800 and a plurality of representative points Pr corresponding to the plurality of voxels VX in FIG. 5(C). The representative points Pr are represented by black dots. The plurality of representative points Pr are arranged along the target region 800A. In the example of FIG. 5(E), the plurality of representative points Pr include a plurality of representative points Pr arranged along the first side E1, a plurality of representative points Pr arranged along the second side E2, and a plurality of representative points Pr arranged along the third side E3.

[0045] For downsampling using the voxel grid method, for example, the Open3D function "voxel_down_sample" may be used. Instead of the voxel grid method, other methods, such as random sampling or farthest point sampling (FPS), may be used.

[0046] In S170 (FIG. 3), the processor 210 prompts the user to register a first target teaching point. The target teaching point is a teaching point that forms a path for machining the workpiece 800 by the tool 990 (i.e., a movement path of the tool 990). In this embodiment, the teaching point indicates the position of the tip 990t of the tool 990 (FIG. 2). The first target teaching point indicates the initial position for machining the workpiece 800.

[0047] Various methods may be used to register the target teaching point. In this embodiment, the processor 210 displays multiple representative points Pr on the display unit 240 ( FIG. 1 ). The processor 210 may display an image representing the workpiece 800 and the multiple representative points Pr on the display unit 240, as shown in FIG. 5(E). The user inputs an instruction to select one representative point Pr from the multiple representative points Pr to the data processing device 200 via the operation unit 250 or the controller 600. The processor 210 calculates the target teaching point using the selected representative point Pr. As described with reference to FIGS. 5(B) to 5(E), the representative point Pr indicates a representative position of point P on the outer surface 800o. The representative point Pr is located near the outer surface 800o. If the representative point Pr were used as the target teaching point as is, the tool 990 located at the target teaching point may interfere with the workpiece 800. In this embodiment, the processor 210 determines the teaching point to be a point away from the representative point Pr in the normal direction of the outer surface 800o.

[0048] FIG. 5(F) shows an example of normal directions. The figure shows a representative point Pr, a sphere SPr centered at the representative point Pr and having a radius Rr, and a normal vector vnr relative to the representative point Pr. In this embodiment, the normal vector vnr is the normal vector of an approximation plane PLr that approximates multiple points P located within the sphere SPr. The approximation plane PLr approximates the outer surface 800o represented by the multiple points P. Various methods may be used to calculate the approximation plane PLr, i.e., the normal vector vnr. The calculation method in this embodiment is as follows: The processor 210 calculates three eigenvectors of the variance-covariance matrix by performing so-called principal component analysis on the multiple points P within the sphere SPr. The three eigenvectors are perpendicular to each other. Although not shown, two of the three eigenvectors (i.e., the first and second principal components) associated with large variances are approximately parallel to the plane represented by the multiple points P (i.e., the outer surface 800o). The approximate plane PLr is a plane parallel to these two eigenvectors. Processor 210 adopts the eigenvector (i.e., the third principal component) associated with the smallest variance among the three eigenvectors as the normal vector vnr. If the direction of the eigenvector is from the representative point Pr toward the inside of workpiece 800, processor 210 determines the direction of the normal vector vnr to be opposite to the direction of the eigenvector.

[0049] The radius Rr may be various values ​​such that an appropriate normal vector vnr can be obtained using a sphere SPr having the radius Rr. The processor 210 may use a predetermined radius Rr. For example, the radius Rr may be half the length of a side of the voxel VX. Alternatively, the processor 210 may determine the radius Rr according to a user instruction.

[0050] FIG. 5(G) shows an example of a first target teaching point and a second target teaching point (described later). The figure shows the same workpiece 800 and multiple representative points Pr as those shown in FIG. 5(E). The representative points Pr are represented by black dots, and the target teaching points Pt are represented by double circles. Corresponding representative points Pr and target teaching points Pt are connected by lines. The first representative point Pr1 is the representative point Pr selected by the user in S170. The first representative point Pr1 is a representative point Pr near the first edge E1. The processor 210 determines the first target teaching point Pt1 at a point away from the first representative point Pr1 by an offset distance in the normal direction of the first representative point Pr1. The offset distance may be any value that allows the target teaching points to form a path suitable for machining the workpiece 800 with the tool 990. The offset distance for the deburring tool 990 may be, for example, several millimeters. The processor 210 may use a predetermined offset distance, or alternatively, the processor 210 may determine the offset distance according to a user instruction.

[0051] As described above, the teaching point represents not only the position of the tool 990 with respect to the workpiece 800 (i.e., the teaching position) but also the orientation of the tool 990 with respect to the workpiece 800 (i.e., the teaching orientation). In this embodiment, the processor 210 determines the orientation of the tool 990 at the teaching point to be the normal direction at the representative point Pr corresponding to the teaching point.

[0052] In response to the determination of the target teaching point, the processor 210 displays the determined target teaching point on the display unit 240 (FIG. 1). The processor 210 may display an image representing the workpiece 800, the plurality of representative points Pr, and the determined target teaching point (here, the first target teaching point Pt1) on the display unit 240, as shown in FIG. 5(G). The user can easily confirm the teaching point by observing the displayed image.

[0053] The teaching point can be expressed in various formats. For example, in S130-S170 (FIG. 3), the processor 210 calculates the teaching point expressed in the camera coordinate system by using a point cloud expressed in the camera coordinate system. Such a teaching point represents three-dimensional coordinates representing the teaching position and three-dimensional components representing the teaching orientation. The workpiece 800 (FIG. 2) and the tool 990 are supported by the support device 930 of the machining apparatus 900. Therefore, the teaching position and teaching orientation can be expressed in a robot coordinate system associated with the machining apparatus 900. The position and orientation of the tool 990 relative to the workpiece 800 are adjusted by the operation of the support device 930 of the machining apparatus 900 (FIG. 2). Therefore, the teaching position and teaching orientation can be expressed by the control parameters of the support device 930. In this embodiment, the control parameters include the sliding positions in the X, Y, and Z directions by the support device 910 and the rotational position by the second support device 920.

[0054] In this embodiment, the processor 210 generates, as teaching point data, data representing a teaching position and teaching orientation in the camera coordinate system, and slide and rotation positions in each of the X, Y, and Z directions. The processor 210 can convert between multiple representation formats of position and orientation as follows. The coordinate correspondence, which is the correspondence between the camera coordinate system and the robot coordinate system, changes depending on the position and orientation of the camera 700 relative to the processing device 900. The coordinate correspondence is experimentally determined in advance before the teaching process, after adjustment of the position and orientation of the camera 700 is completed (details will be described later). Furthermore, the control parameter correspondence, which is the correspondence between the robot coordinate system and control parameters (slide position and rotation position), is experimentally determined in advance before the teaching process. For example, the control parameter correspondence can be determined by actually measuring the three-dimensional coordinates of the tip 990t of the tool 990 corresponding to various control parameters. The processor 210 can convert the representation formats of position and orientation by using the coordinate correspondence and the control parameter correspondence.

[0055] Various methods may be used to determine the coordinate correspondence relationship. For example, a reference object (e.g., an AR marker) is placed within the reading range of the camera 700. The reference object includes multiple reference portions that define a coordinate system (e.g., a portion indicating the origin, a line indicating the X direction, and a line indicating the Y direction). The camera 700 reads the reference object and acquires the three-dimensional coordinates of each of the multiple reference portions in the camera coordinate system. The processing device 900 measures the three-dimensional coordinates of each of the multiple reference portions in the robot coordinate system. For example, the processing device 900 can acquire the three-dimensional coordinates in the robot coordinate system by moving the tip 990t of the tool 990 to the position of the reference portion. A user can determine the coordinate correspondence relationship using the correspondence relationship between the three-dimensional coordinates of the multiple reference portions in the camera coordinate system and the three-dimensional coordinates in the robot coordinate system. Note that the camera 700 may read a reference object in addition to the workpiece 800 in S120 (FIG. 3). The processor 210 may determine the coordinate correspondence relationship using information about the read reference object. The processor 210 may determine the coordinate correspondence at any time prior to the process in which the coordinate correspondence is used.

[0056] In this embodiment, the total number of rotation axes usable by the machining device 900 ( FIG. 2 ) to change the orientation of the tool 990 is one, not two. Therefore, it may be difficult to precisely align the orientation of the tool 990 with respect to the workpiece 800 to the taught orientation. In this embodiment, the processor 210 calculates an approximate solution of the rotation position of the second support device 920 corresponding to the taught orientation. The approximate solution may be various rotation positions that bring the orientation of the tool 990 close to the taught orientation. For example, the processor 210 may calculate the rotation position as follows: When the table 920t rotates from the rotation position in S120 ( FIG. 3 ), the taught position and taught orientation rotate together with the table 920t around the rotation axis Da. The processor 210 calculates a rotation position (referred to as a target rotation position) at which the angle between the actual orientation of the tool 990 (here, the +Z direction) and the taught orientation is the smallest. The target rotation position is an example of an approximate solution of the rotation position. The processor 210 may determine the target rotational position by calculating the angle between the taught position and the taught orientation at various rotational positions.

[0057] The processor 210 calculates slide positions in each of the X, Y, and Z directions for moving the tip 990t of the tool 990 to the taught position at the target rotation position (referred to as target slide positions). As a result, the processor 210 can acquire the target slide positions in each of the X, Y, and Z directions and the target rotation position that are associated with the taught point.

[0058] In S170 (FIG. 3), the processor 210 stores the acquired data of the first target teaching point Pt1 in the storage device 215 (for example, the nonvolatile storage device 230). The stored data of the first target teaching point Pt1 represents the teaching position and teaching orientation in the camera coordinate system, the target slide positions in X, Y, and Z, and the target rotation position.

[0059] In S180, processor 210 prompts the user to register a second target teaching point. The second target teaching point indicates the teaching point next to first target teaching point Pt1. The processing of S180 is similar to the processing of S170. FIG. 5(G) shows second representative point Pr2 and second target teaching point Pt2. Second representative point Pr2 is the representative point selected by the user in S180. Processor 210 places second target teaching point Pt2 at a point away from second representative point Pr2 by the offset distance in the normal direction of second representative point Pr2.

[0060] In S210 (FIG. 4), the processor 210 executes a process for determining a candidate teaching point cPt. The candidate teaching point cPt is a candidate for the next target teaching point. FIG. 6 is a flowchart showing an example of the process for determining a candidate teaching point. In S310, the processor 210 calculates a three-dimensional vector Vs based on the two most recent target teaching points Pta and Ptb. The three-dimensional vector Vs indicates an estimated direction in which the target area 800A extends. In this embodiment, the processor 210 determines the three-dimensional vector Vs to be a vector directed from the representative point Pra associated with the second most recent target teaching point Pta to the representative point Prb associated with the most recent target teaching point Ptb (Vs=Prb-Pra).

[0061] In S315, the processor 210 searches for a representative point Pr within a three-dimensional specific range VRs. The specific range VRs is a three-dimensional region extending from the representative point Prb associated with the latest target teaching point Ptb in the direction of the three-dimensional vector Vs. FIG. 7A is a diagram illustrating an example of the specific range VRs. The diagram illustrates the target teaching points Pta and Ptb, the representative points Pra and Prb, and the three-dimensional vector Vs. In this embodiment, the shape of the specific range VRs is a cylinder centered on a line L passing through the representative points Pra and Prb. The radius Rs centered on the line L and the length Ls in the direction of the three-dimensional vector Vs are experimentally determined in advance so that the next appropriate representative point Pr after the latest representative point Prb is included in the specific range VRs. Alternatively, the processor 210 may determine the radius Rs and the length Ls in accordance with a user instruction.

[0062] In the example of FIG. 7(A), the specific range VRs includes a portion 800op of the outer surface 800o. Hereinafter, this portion 800op will be referred to as the specific portion 800op. The specific range VRs includes a representative point Pr associated with the specific portion 800op. Note that in the drawing, the specific portion 800op is simplified and represented by a plane. In reality, the shape of the specific portion 800op may be various shapes.

[0063] In S320 (FIG. 6), the processor 210 determines whether the total number of representative points Pr within the specific range VRs is greater than the first threshold value Th1. If the target region 800A extends from a vicinity of the most recent representative point Prb in the direction of the three-dimensional vector Vs, the total number of representative points Pr included in the specific range VRs may be large. In this case, the processor 210 can select an appropriate representative point Pr for the next candidate teaching point cPt from within the specific range VRs. The extension direction of the target region 800A may change along the way. That is, the extension direction of the target region 800A may differ from the direction of the three-dimensional vector Vs. If the target region 800A extends in a direction different from the direction of the three-dimensional vector Vs, the total number of representative points Pr included in the specific range VRs may be small. In this case, the processor 210 may not be able to select an appropriate representative point Pr for the next candidate teaching point cPt from within the specific range VRs.

[0064] The first threshold value Th1 is experimentally determined in advance so that the total number of representative points Pr is greater than the first threshold value Th1 when the target region 800A extends in the direction of the three-dimensional vector Vs. For example, the first threshold value Th1 may be zero. Alternatively, the first threshold value Th1 may be a value greater than or equal to one. The processor 210 may determine the first threshold value Th1 in accordance with a user instruction.

[0065] If the total number of representative points Pr within the specific range VRs is greater than the first threshold value Th1 (S320: Yes), in S325, the processor 210 selects the representative point Pr closest to the latest representative point Prb from among the representative points Pr within the specific range VRs. The processor 210 determines the candidate teaching point cPt as the teaching point associated with the selected representative point Pr. In the example of FIG. 7(A), it is assumed that the determination result in S320 is Yes. The processor 210 determines the candidate teaching point cPt using the representative point cPr closest to the latest representative point Prb.

[0066] The method for determining the candidate teaching point cPt from the representative point cPr is the same as the method described in S170 (FIG. 3). FIG. 7(B) is a diagram showing an example of the candidate teaching point cPt. As in the example of FIG. 5(F), the processor 210 calculates the normal vector vnc of the approximation plane cPL using point P within a sphere (not shown) having a radius Rr and centered at the representative point cPr. The processor 210 determines the position of the candidate teaching point cPt to be a position moved by an offset distance dn from the representative point cPr in the direction of the normal vector vnc. The movement vector MV1 in the diagram is a vector indicating the movement from the representative point cPr to the candidate teaching point cPt. The movement vector MV1 includes a component in the normal direction at the representative point cPr. This normal direction is the normal direction to the approximation plane cPL, i.e., the normal direction to the outer surface 800o. After S325 (FIG. 6), the processor 210 proceeds to S350.

[0067] FIG. 7C is a diagram illustrating an example of multiple representative points Pr and a specific range VRs. FIG. 7C illustrates an example in which the total number of representative points Pr included in the specific range VRs is equal to or less than a first threshold value Th1. As will be described later, target teaching points are added one by one through the teaching process. In the example of FIG. 7C, it is assumed that target teaching points Pt1-Pt6 corresponding to representative points Pr1-Pr6 are determined in this order. The path PTH in the figure is the path formed by the target teaching points Pt1-Pt6 (the path PTH is a portion of the path finally determined). The latest representative point Pr6 is located at the end of the first side E1. The target region 800A branches from this point into a portion extending along the second side E2 and a portion extending along the third side E3. The specific range VRs in the figure extends from the latest representative point Pr6 in the direction of the three-dimensional vector Vs. The extension direction of this specific range VRs is approximately the same as the extension direction of the first side E1. On the other hand, the extension direction of the specific range VRs is different from the extension direction of the second side E2 and the extension direction of the third side E3. No representative point Pr exists within this specific range VRs. In this way, when the extension direction of the target region 800A is different from the direction of the three-dimensional vector Vs, the total number of representative points cPr may be less than or equal to the first threshold Th1.

[0068] If the total number of representative points Pr within the specific range VRs is less than or equal to the first threshold Th1 (FIG. 6: S320: No), in S330, the processor 210 searches for a representative point cPr within a neighborhood range VRn centered on the most recent representative point Prb. FIG. 7(D) is a diagram showing an example of the neighborhood range VRn. The neighborhood range VRn is a three-dimensional region contained in a sphere SPn having a radius Rn and centered on the most recent representative point Prb. In FIG. 7(D), it is assumed that the sixth representative point Pr6 shown in FIG. 7(C) is the most recent representative point Prb. As shown, the neighborhood range VRn may include representative points Pr located in various directions relative to the most recent representative point Prb. In the example of FIG. 7(D), the neighborhood range VRn includes a representative point Pr near the second edge E2 and a representative point Pr near the third edge E3.

[0069] In S335 (FIG. 6), the processor 210 determines whether the total number of representative points Pr within the vicinity range VRn is greater than the second threshold value Th2. Here, the representative points Pr associated with the target teaching points are excluded. For example, in the example of FIG. 7(D), the representative points Pr4-Pr6 are associated with the target teaching points Pt4-Pt6, respectively, and are therefore excluded from the calculation of the total number of representative points Pr. If the target region 800A extends further from the vicinity of the latest representative point Prb, the total number of representative points Pr included in the vicinity range VRn may be large. In this case, the processor 210 can select an appropriate representative point Pr for the next candidate teaching point cPt from within the vicinity range VRn. As the teaching process progresses along the target region 800A, the latest representative point Prb may reach the edge of the target region 800A. If the latest representative point Prb is located at the edge of the target region 800A, the total number of representative points Pr included in the vicinity range VRn may be small. In this case, the processor 210 may not be able to select an appropriate representative point Pr for the next candidate teaching point cPt from within the neighborhood range VRn.

[0070] The second threshold value Th2 is experimentally determined in advance so that the total number of representative points Pr is greater than the second threshold value Th2 when the target region 800A extends further. For example, the second threshold value Th2 may be zero. Alternatively, the second threshold value Th2 may be a value greater than or equal to one. The processor 210 may determine the second threshold value Th2 in accordance with a user instruction.

[0071] If the total number of representative points Pr within the vicinity range VRn is greater than the second threshold value Th2 (S335: Yes), in S340, the processor 210 selects the representative point Pr closest to the latest representative point Prb from among the representative points Pr within the vicinity range VRn (excluding the representative point Pr associated with the target teaching point). The processor 210 determines the candidate teaching point cPt as the teaching point associated with the selected representative point Pr. In the example of FIG. 7(D), it is assumed that the determination result in S335 is Yes. The processor 210 determines the candidate teaching point cPt using the representative point cPr closest to the latest representative point Prb. In the example of FIG. 7(D), the representative point Pr near the third side E3 is selected as the representative point cPr. After S340, the processor 210 proceeds to S350.

[0072] If the total number of representative points Pr in the vicinity range VRn is equal to or less than the second threshold value Th2 (S335: No), in S345, processor 210 sets the candidate teaching point cPt to none. Then, processor 210 proceeds to S350.

[0073] In S350, when the candidate teaching point cPt is found, the processor 210 estimates the normal direction at the candidate teaching point cPt. In this embodiment, the processor 210 determines the normal direction at the candidate teaching point cPt to be the same direction as the direction of the normal vector vnc at the representative point cPr associated with the candidate teaching point cPt.

[0074] In S355, if the candidate teaching point cPt is found, the processor 210 determines the orientation of the tool 990 at the candidate teaching point cPt to be the normal direction at the candidate teaching point cPt (i.e., the direction of the normal vector vnc at the representative point cPr corresponding to the candidate teaching point cPt). As in S170 (FIG. 3), the processor 210 determines the teaching position and teaching orientation in the camera coordinate system, the slide positions in X, Y, and Z, and the rotation position, which are associated with the candidate teaching point cPt. Then, the processor 210 stores the data of the candidate teaching point cPt in the storage device 215 (e.g., the non-volatile storage device 230). Then, the processor 210 ends the processing of FIG. 6, i.e., the processing of S210 in FIG. 4.

[0075] If the candidate teaching point cPt is set to none in S210, the processor 210 proceeds to S260. If the candidate teaching point cPt is found, the processor 210 proceeds to S215.

[0076] In S215, the processor 210 presents the candidate teaching points cPt to the user. In this embodiment, the processor 210 controls the machining apparatus 900 to adjust the position and orientation of the tool 990 relative to the workpiece 800 to the position and orientation represented by the candidate teaching points cPt. In this embodiment, the processor 210 supplies the machining apparatus 900 with instructions to control the support device 930 according to the X, Y, and Z sliding positions and rotational positions represented by the candidate teaching points cPt. The control device 980 of the machining apparatus 900 controls the support devices 930 (here, the first support device 910 and the second support device 920) according to the instructions.

[0077] After S215, the user considers whether the position and orientation of the tool 990 relative to the workpiece 800 are appropriate by observing the workpiece 800 and the tool 990. In S225, the user operates the operation unit 250 or the controller 600 to input an adoption instruction indicating adoption of the candidate teaching point cPt or a correction instruction indicating correction of the candidate teaching point cPt to the data processing device 200. The processor 210 acquires the input instruction.

[0078] In S230, the processor 210 determines whether the instruction is an adoption instruction. If the instruction is an adoption instruction (S230: Yes), in S248, the processor 210 determines the new target teaching point Pt as the candidate teaching point cPt. Then, the processor 210 stores data of the determined target teaching point Pt in the storage device 215 (e.g., the non-volatile storage device 230). Then, the processor 210 proceeds to S260.

[0079] If the instruction differs from the adopted instruction (S230: No), in S253, the user manually corrects the position and orientation of the tool 990 relative to the workpiece 800 to an appropriate position and orientation. FIG. 7(E) is a diagram showing an example of candidate teaching point cPt and corrected candidate teaching point cPtm. The figure shows the same representative points Pr5-Pr6, target teaching points Pt5-Pt6, representative points cPr, and candidate teaching point cPt as those in FIG. 7(D). If the candidate teaching point cPt is inappropriate, the user may manually correct the candidate teaching point cPt to the candidate teaching point cPtm. In the example of FIG. 7(E), the corrected candidate teaching point cPtm is located near the second edge E2.

[0080] Various methods may be used to manually correct the position and orientation. For example, a user may input an instruction to change one or both of the position and orientation by operating the operation unit 250 or the controller 600 of the data processing device 200 (FIG. 1). For example, the processor 210 may receive an instruction regarding the direction and amount of movement. The processor 210 may also receive an instruction regarding the direction and amount of change in orientation. The instruction may be represented, for example, by control parameters (slide position and rotation position) of the processing device 900. The processor 210 provides the change instruction to the processing device 900 in accordance with the input instruction. The control device 980 of the processing device 900 changes the position and / or orientation by controlling the support device 930 (here, the first support device 910 and the second support device 920) in accordance with the change instruction. Alternatively, the user may move one or both of the table 920t and the tool 990 by applying a force to one or both of the table 920t and the tool 990. As a result, one or both of the position and the orientation change. After completing the correction, the user inputs a completion instruction to the data processing device 200 via the operation unit 250 or the controller 600.

[0081] In S258, the processor 210 requests the current slide positions in the X, Y, and Z directions and the rotation position from the machining device 900. The control device 980 of the machining device 900 supplies the requested information to the data processing device 200. The processor 210 uses the acquired information to generate data of the corrected candidate teaching point cPtm. Here, the teaching position and teaching orientation in the camera coordinate system may be omitted. The processor 210 determines the new target teaching point Pt as the corrected candidate teaching point cPtm. Then, the processor 210 stores the data of the determined target teaching point Pt in the storage device 215 (e.g., the non-volatile storage device 230).

[0082] In S258, the processor 210 further calculates a corrected representative point cPrm associated with the corrected candidate teaching point cPtm (FIG. 7(E)). The processor 210 stores data of the corrected representative point cPrm in the storage device 215 (e.g., non-volatile storage device 230). The corrected representative point cPrm is used as a representative point Pr associated with the target teaching point Pt in the next S210 (FIG. 4). The corrected representative point cPrm is a point on the target region 800A of the workpiece 800, and may be any of various points in the vicinity of the corrected candidate teaching point cPtm. For example, the processor 210 may select, as the corrected representative point cPrm, a point P that is closest to the corrected candidate teaching point cPtm from the point cloud of a specific color (FIG. 3: S140, FIG. 5(B)). Here, the processor 210 uses the slide positions and rotation positions in the X, Y, and Z directions of the corrected candidate teaching point cPtm to obtain the position and orientation in the robot coordinate system. The processor 210 uses the position and orientation in the robot coordinate system to obtain the position and orientation in the camera coordinate system. The processor 210 can use the position in the camera coordinate system to search for the point P closest to the corrected candidate teaching point cPtm.

[0083] Alternatively, the processor 210 may determine the corrected representative point cPrm as follows: The processor 210 calculates a normal at each point P of the point cloud of a specific color. The normal at point P is a straight line that passes through point P and extends in the direction of the normal vector at point P. The method of calculating the normal vector at point P is the same as the method described in FIG. 5(F). The processor 210 selects, from among multiple normals, the normal that is closest to the corrected candidate teaching point cPtm. The processor 210 may select point P associated with the selected normal as the corrected representative point cPrm. The processor 210 can perform these processes using a position in the camera coordinate system.

[0084] After S258, the processor 210 proceeds to S260.

[0085] In S260, the processor 210 determines whether or not an unprocessed representative point Pr may remain in the sampled point cloud. If a candidate teaching point cPt is found in the final S210, the processor 210 determines that an unprocessed representative point Pr may remain. In this case (S260: Yes), the processor 210 proceeds to S210 and executes processing for the next target teaching point Pt. If the candidate teaching point cPt is set to none in the final S210, the processor 210 determines that no unprocessed representative point Pr remains. In this case (S260: No), the processor 210 proceeds to S265.

[0086] In S265, the processor 210 determines a candidate approach point cPtS using the position and normal direction of the first target teaching point Pt1. The approach point is a type of teaching point, and represents the position and orientation of the tool 990 to reach before starting machining of the workpiece 800. The approach point may be any of various teaching points that the tool 990 can move from to the first target teaching point Pt1 without interfering with the workpiece 800.

[0087] FIG. 7(F) is a diagram illustrating an example of the candidate approach point cPtS. The diagram illustrates a first target teaching point Pt1, a first representative point Pr1 associated with the first target teaching point Pt1, a normal vector vn1 at the first representative point Pr1, and an approximate plane PL1 associated with the normal vector vn1. In this embodiment, the processor 210 determines the candidate approach point cPtS at a point separated from the first target teaching point Pt1 by a first distance dS in the direction of the normal vector vn1. The processor 210 determines the orientation of the tool 990 at the candidate approach point cPtS to be the normal direction of the first target teaching point Pt1 (i.e., the direction of the normal vector vn1 at the first representative point Pr1). The first distance dS may be any value that indicates that the candidate approach point cPtS is sufficiently distant from the workpiece 800. The processor 210 may use a predetermined first distance dS. Alternatively, the processor 210 may determine the first distance dS according to a user instruction.

[0088] As in S170 (FIG. 3), processor 210 determines the X, Y, and Z slide positions and the rotation position associated with candidate approach point cPtS. Note that calculation of the teaching position and teaching orientation in the camera coordinate system may be omitted.

[0089] In S270 (FIG. 4), the processor 210 presents the candidate approach points cPtS to the user in the same manner as in S215.

[0090] In S275, the processor 210 determines the approach point PtS by adopting or modifying the candidate approach point cPtS. The processing of S275 is performed in the same manner as the adoption or modification of the candidate teaching point cPt in S225-S258. Note that calculation of the representative point associated with the candidate approach point is omitted.

[0091] In S280, the processor 210 determines a candidate retreat point cPtE using the position and normal direction of the last target teaching point PtN. The retreat point is a type of teaching point, and represents the position and orientation of the tool 990 to reach after finishing machining the workpiece 800. The retreat point may be any of various teaching points that the tool 990 can move to from the last target teaching point PtN without interfering with the workpiece 800.

[0092] FIG. 7(G) is a diagram illustrating an example of the candidate retreat point cPtE. The diagram illustrates the last target teaching point PtN, the last representative point PrN associated with the last target teaching point PtN, a normal vector vnN at the last representative point PrN, and an approximate plane PLN associated with the normal vector vnN. In this embodiment, the processor 210 determines the candidate retreat point cPtE at a point separated from the last target teaching point PtN by a second distance dE in the direction of the normal vector vnN. The processor 210 determines the orientation of the tool 990 at the candidate retreat point cPtE to be the normal direction of the last target teaching point PtN (i.e., the direction of the normal vector vnN at the last representative point PrN). The second distance dE may be any value that indicates that the candidate retreat point cPtE is sufficiently far from the workpiece 800. The processor 210 may use a predetermined second distance dE. Alternatively, the processor 210 may determine the second distance dE according to instructions from the user.

[0093] The processor 210 determines the X, Y, and Z slide positions and the rotation position associated with the candidate retreat point cPtE, similarly to S265 (FIG. 4). Note that the calculation of the teaching position and teaching orientation in the camera coordinate system may be omitted.

[0094] In S285, the processor 210 presents the candidate save point cPtE to the user in the same manner as in S215.

[0095] In S290, the processor 210 determines the save point PtE by adopting or modifying the candidate save point cPtE. The process of S290 is performed in the same manner as in S275.

[0096] In S295, the processor 210 stores teaching point set data, which is data representing a set of the approach point PtS, multiple target teaching points Pt, and the retreat point PtE, in the storage device 215 (for example, the non-volatile storage device 230). The teaching point set data represents multiple target teaching points Pt determined by the teaching process. Then, the processor 210 ends the teaching process of FIGS. 3 and 4.

[0097] As described above, the processor 210 determines a plurality of sequentially arranged teaching points PtS, Pt, and PtE through the teaching process. The determined plurality of teaching points includes a plurality of sequentially arranged target teaching points Pt. The plurality of target teaching points Pt represent a machining path that sequentially passes through the plurality of target teaching points Pt. After the teaching process, the processor 210 may actually move the tool 990 along a path that sequentially passes through the approach point PtS, the plurality of target teaching points Pt, and the retreat point PtE. The user can confirm whether an appropriate path is formed by observing the tool 990 actually moving. The teaching point set data may also be used for machining a new workpiece 800.

[0098] Furthermore, there may be cases where the tool 990 is away from the workpiece 800 at the target teaching point Pt. The processor 210 may update the target teaching point Pt as follows. At each target teaching point Pt, the processor 210 moves the tool 990 in the direction opposite to the normal direction (i.e., the direction approaching the workpiece 800) until the tool 990 comes into contact with the workpiece 800. Then, the processor 210 may adopt the position where the tool 990 comes into contact with the workpiece 800 as the updated target teaching point Pt. The processor 210 stores teaching point set data representing the updated target teaching point Pt in the storage device 215.

[0099] As described above, in this embodiment, the processor 210 executes the following processing in accordance with the program 231. In S215 (FIG. 4), the processor 210 presents the candidate teaching point cPt, which is a candidate for the target teaching point Pt, to the user (hereinafter, the processing of S215 will be referred to as presentation processing S215). The target teaching point Pt is a teaching point that forms a path for machining the workpiece 800 by the tool 990. The workpiece 800 is an example of an object to be machined. In S225, S230, and S248, the processor 210 determines a new target teaching point Pt in accordance with an instruction from the user. In this embodiment, the teaching point determination processing for determining a new target teaching point Pt includes the processing of S225, S230, and S248. The processes of S225, S230, and S248 include a process (S248) of determining a new target teaching point Pt as a presented candidate teaching point cPt when the user's instruction is an adoption instruction (S230: Yes). The adoption instruction is an instruction indicating adoption of the presented candidate teaching point cPt. In S210, the processor 210 determines the next candidate teaching point cPt to be presented based on one or more target teaching points including the new target teaching point Pt (i.e., the latest target teaching point Ptb) (hereinafter, the process of S210 will be referred to as next candidate determination process S210). As described with reference to FIG. 6, in this embodiment, the latest two target teaching points Pta and Ptb are used in the next candidate determination process S210.

[0100] The processor 210 determines a plurality of target teaching points Pt by repeating a process including S210, S215, S225, S230, and S248 (hereinafter, the process of repeating the process including S210, S215, S225, S230, and S248 is referred to as a determination process PD). The determination process PD includes a process of adding new target teaching points Pt one by one to the plurality of target teaching points Pt by repeating the presentation process S215, the teaching point determination process (including S225, S230, and S248), and the next candidate determination process S210.

[0101] In this way, the processor 210 presents the user with candidate teaching points cPt that are candidates for the target teaching point Pt, and when the user's instruction is an adoption instruction, determines a new target teaching point Pt as the presented candidate teaching point cPt, and determines the next candidate teaching point cPt to be presented based on one or more target teaching points including the new target teaching point Pt (i.e., the latest target teaching point Ptb). By repeating this process, the processor 210 adds new target teaching points Pt one by one to the multiple target teaching points Pt. In this way, the processor 210 presents the candidate teaching points cPt to the user, and when the user's instruction is an adoption instruction, determines the new target teaching point Pt as the presented candidate teaching point cPt. Compared to when the user sets the candidate teaching point cPt himself, the influence of the user's proficiency on the candidate teaching points cPt (and therefore the target teaching point Pt) is mitigated. The processor 210 can reduce the variation in the position of the target teaching point Pt. Furthermore, since the processor 210 presents the candidate teaching points cPt to the user, the user does not need to set the candidate teaching points cPt by himself / herself. In this way, the processor 210 can reduce the burden on the user.

[0102] Furthermore, in this embodiment, the target teaching point Pt represents not only the position but also the orientation of the tool 990 relative to the workpiece 800. The processor 210 determines both the position and the orientation of the tool 990 relative to the workpiece 800. This eliminates the need for the user to set the orientation of the tool 990 by themselves. This allows the processor 210 to reduce the burden on the user.

[0103] In this embodiment, the process of determining a new target teaching point Pt in accordance with a user instruction includes S225, S230, S253, and S258. In this embodiment, the teaching point determination process of determining a new target teaching point Pt includes the processes of S225-S258. The processes of S225-S258 include processes (S253, S258) of determining the new target teaching point as a modified teaching point (e.g., candidate teaching point cPtm ( FIG. 7(E))) when the user instruction is a correction instruction (S230: No). The correction instruction is an instruction indicating correction of the presented candidate teaching point. In this way, the processor 210 allows the user to correct the teaching point. Therefore, the processor 210 can reduce the possibility that an inappropriate target teaching point will be included among the multiple target teaching points Pt.

[0104] In this embodiment, as described in FIG. 5(A), the workpiece 800 has an outer surface 800o including a target area 800A. As described in S110 (FIG. 3), the target area 800A is formed by the user. The target area 800A is an example of a specified area. The next candidate determination process S210 (FIG. 4) includes S325 of FIG. 6. In S325, the processor 210 determines the candidate teaching point cPt as a teaching point associated with the representative point Pr. As described in S160 (FIG. 3), the representative point Pr is acquired using a point cloud of a specific color, i.e., a point cloud associated with the target area 800A. In this way, in S325, the processor 210 determines the candidate teaching point cPt to be presented next as a teaching point associated with the specified area (here, the target area 800A). As a result of the above, the processor 210 can determine a plurality of target teaching points that form a path suitable for machining the target region 800A of the workpiece 800.

[0105] In this embodiment, the workpiece 800 (FIG. 5A) has an outer surface 800o. The next candidate determination process S210 includes S310, S315, and S325 of FIG. 6. In S310, the processor 210 calculates a three-dimensional vector Vs based on a plurality of target teaching points (in this embodiment, target teaching points Pta and Ptb) including the latest target teaching point Ptb (FIG. 7A). In S325, the processor 210 determines, as shown in FIG. 7A, the candidate teaching point cPt to be presented next as a teaching point associated with a representative point Pr associated with a specific portion 800op, which is part of the outer surface 800o of the workpiece 800. In this way, the candidate teaching point cPt is a teaching point associated with the specific portion 800op. As shown in FIG. 7A, the specific portion 800op is a portion within a three-dimensional specific range VRs extending in the direction of the three-dimensional vector Vs from a representative point Prb, which is a point on the outer surface 800o associated with the latest target teaching point Ptb. The representative point Prb is an example of a reference point, which is a point on the outer surface 800o associated with the latest target teaching point Ptb. Such a specific portion 800op is likely to include a portion of the outer surface 800o that should be machined following the latest target teaching point Ptb. In other words, the candidate teaching point cPt associated with the specific portion 800op is likely to be suitable as the target teaching point Pt following the latest target teaching point Ptb. The processor 210 can determine an appropriate candidate teaching point cPt in this manner.

[0106] In this embodiment, the next candidate determination process S210 (FIG. 4) includes S325 (FIG. 6). In S325, the processor 210 determines the next candidate teaching point cPt (FIGS. 7A and 7B) to be presented as a teaching point located at a position obtained by moving the representative point cPr in a first direction (here, the direction of the movement vector MV1). The representative point cPr is a point acquired by downsampling the point cloud measured by the camera 700, as described in S160 (FIG. 3). The point P included in the downsampled point cloud indicates a point on the outer surface 800o of the workpiece 800 and represents a measured value in three-dimensional coordinates. Thus, the representative point cPr is an example of a first point, which is a point on the outer surface 800o of the workpiece 800 estimated using the measured value in three-dimensional coordinates. As described in FIG. 7B, the direction of the movement vector MV1 includes a component in the normal direction of the outer surface 800o at the representative point cPr. In this way, the processor 210 can determine the candidate teaching point cPt to be presented next as a teaching point at a position that does not interfere with the workpiece 800.

[0107] Furthermore, in this embodiment, the processor 210 executes the following processing. In S265 (FIG. 4), the processor 210 determines a candidate approach point cPtS that is a candidate for the approach point PtS, which is the teaching point immediately preceding the first target teaching point Pt1 among the multiple target teaching points Pt. In S270, the processor 210 presents the candidate approach point cPtS to the user. In S275, the processor 210, in accordance with a user instruction, determines the approach point PtS to be the candidate approach point cPtS or a teaching point designated by the user. In S280, the processor 210 determines a candidate retreat point cPtE that is a candidate for the retreat point PtE, which is the teaching point immediately following the last target teaching point PtN among the multiple target teaching points Pt. In S285, the processor 210 presents the candidate retreat point cPtE to the user. In S290, the processor 210 determines the retreat point PtE to be the candidate retreat point cPtE or a teaching point designated by the user in accordance with a user instruction. In this manner, the processor 210 can determine a plurality of teaching points that form the path of the tool 990 from before the start of machining to after the end of machining.

[0108] B. Second Example: FIG. 8 is a part of a flowchart showing another embodiment of the teaching process. FIG. 8 shows the process following S180 (FIG. 3) instead of the process of FIG. 4. In this embodiment, the processor 210 (FIG. 1) uses the display unit 240 to present the candidate teaching points cPt to the user. S210 (FIG. 8) following S180 is the same as S210 in FIG. 4. The processor 210 executes the process of determining the candidate teaching points cPt.

[0109] If the candidate teaching point cPt is set to none in S210, the processor 210 proceeds to S260. If the candidate teaching point cPt is found, the processor 210 proceeds to S215b.

[0110] In S215b, the processor 210 presents the candidate teaching points cPt to the user. In this embodiment, the processor 210 displays on the display unit 240 a reference image representing the candidate teaching points cPt.

[0111] FIG. 9 is a diagram illustrating an example of an image displayed on the display unit 240. In this embodiment, the processor 210 displays two windows WNa and WNb on the display unit 240. The first window WNa represents a reference image IMr. The reference image IMr includes an image of the workpiece 800, an image of the target teaching point Pt (here, a double circle), an image of the representative point Pr associated with the target teaching point Pt (here, a black dot), an image of the candidate teaching point cPt (here, a large double circle), an image of the representative point cPr associated with the candidate teaching point cPt (here, a large black dot and a circle), an image of the tool 990 positioned at the candidate teaching point cPt (here, a rectangle), and an image showing the rotation axis AXi (here, a dotted line). The rotation axis AXi represents the rotation axis of the tool 990 when changing the orientation of the tool 990 relative to the workpiece 800. In this embodiment, the rotation axis AXi is represented by a straight line passing through the position of the candidate teaching point cPt. Rotation of the tool 990 around the rotation axis AXi (i.e., change of orientation) is performed by a combination of movement by the support device 910 and rotation by the second support device 920, as described in Fig. 2. The processor 210 uses the image of the workpiece 800 in the two-dimensional image acquired in S120 (Fig. 3) as the image of the workpiece 800. A previously prepared image is used as the image of the tool 990.

[0112] The processor 210 can calculate the positions of each of the points Pt, Pr, cPt, and cPr on the image of the workpiece 800 as follows. As described above, in S210 (FIG. 8), the processor 210 acquires the taught position and taught orientation of each of the points Pt, Pr, cPt, and cPr in the camera coordinate system, as well as the slide positions in X, Y, and Z, and the rotation position. In S120 and S130 (FIG. 3), the processor 210 acquires the three-dimensional coordinates of each of the multiple points P on the two-dimensional image in the camera coordinate system. The multiple points P represent the correspondence between the three-dimensional coordinates in the camera coordinate system and the positions on the two-dimensional image including the image of the workpiece 800. The processor 210 can calculate the position of the workpiece 800 on the two-dimensional image from the three-dimensional coordinates of each of the points Pt, Pr, cPt, and cPr in the camera coordinate system by interpolation using the multiple points P. The same applies to the orientation of the tool 990 on the two-dimensional image of the workpiece 800. The processor 210 can calculate the orientation of the workpiece 800 on the two-dimensional image from the orientation in the camera coordinate system by interpolation using multiple points P.

[0113] The second window WNb displays a first slider SD1, a second slider SD2, and an end button BT. The first slider SD1 is a slider for changing the orientation of the tool 990 relative to the workpiece 800. The second slider SD2 is a slider for adjusting the display size of a portion of the reference image IMr that includes the candidate teaching points cPt. When the second slider SD2 is operated via the operation unit 250, the processor 210 enlarges or reduces the portion of the reference image IMr that includes the candidate teaching points cPt and displays it in the first window WNa. The end button BT is a button for ending the confirmation of the candidate teaching points cPt.

[0114] After S215b (FIG. 8), the user considers whether the candidate teaching point cPt (i.e., the position and orientation of the tool 990 relative to the workpiece 800) is appropriate by observing the reference image IMr. If the candidate teaching point cPt is inappropriate, the user can change the position of the candidate teaching point cPt by moving the candidate teaching point cPt in the reference image IMr via the operation unit 250. For example, if the operation unit 250 includes a mouse, the user may move the candidate teaching point cPt by dragging the image of the candidate teaching point cPt in the reference image IMr using the mouse (the processor 210 calculates a new position in response to the operation of the operation unit 250). The user can also change the orientation of the machining device 900 relative to the workpiece 800 by operating the first slider SD1 via the operation unit 250 (the processor 210 calculates a new orientation in response to the operation of the first slider SD1).

[0115] In S225b, the user operates the operation unit 250 to input, into the data processing device 200, an adoption instruction indicating the adoption of the candidate teaching point cPt or a correction instruction indicating the correction of the candidate teaching point cPt. The processor 210 acquires the input instruction. In this embodiment, the processor 210 treats an instruction to change the position of the candidate teaching point cPt and an instruction to operate the first slider SD1 (i.e., an instruction to change the orientation) as a correction instruction. When the end button BT is operated without an instruction to change the position or an instruction to change the orientation, the processor 210 treats the operation of the end button BT as an adoption instruction.

[0116] In S230b, the processor 210 determines whether the instruction is an adoption instruction. If the instruction is an adoption instruction (S230b: Yes), the processor 210 proceeds to S248. The processing of S248 is the same as the processing of S248 in FIG. 4. The processor 210 determines the new target teaching point Pt as the candidate teaching point cPt, and stores the data of the target teaching point Pt in the storage device 215 (for example, the non-volatile storage device 230). Then, the processor 210 proceeds to S260.

[0117] If the instruction differs from the adopted instruction (S230b: No), in S253b, the user manually corrects the position and orientation of the tool 990 relative to the workpiece 800 to an appropriate position and orientation. As described above, the user can correct the position and orientation by operating the operation unit 250. The processor 210 corrects the X, Y, and Z slide positions and the rotation position in accordance with the input instruction. Although not shown, the processor 210 moves the image of the candidate teaching point cPt, the image of the rotation axis AXi, and the image of the tool 990 in the reference image IMr to the corrected positions. The processor 210 changes the orientation of the image of the tool 990 in the reference image IMr according to the corrected orientation. For example, as shown in FIG. 9, the tool 990 in the reference image IMr rotates in response to the operation of the first slider SD1. The processor 210 also calculates a corrected representative point associated with the corrected candidate teaching point cPt. The calculation method is the same as the calculation method of the corrected representative point cPrm described in S258 of Fig. 4. The processor 210 displays the corrected representative point in the reference image IMr. In this way, the processor 210 changes the position and orientation of the tool 990 in the displayed reference image IMr in accordance with the instruction.

[0118] Note that it may not be easy to change the three-dimensional position in the reference image IMr. In this case, the processor 210 may display an image of multiple representative points Pr on the display unit 240, as shown in FIG. 7(C), and allow the user to select one representative point Pr. The processor 210 may use the selected representative point Pr to calculate a corrected candidate teaching point. In this case, the processor 210 can calculate the position of the corrected candidate teaching point on the reference image IMr as follows: The processor 210 uses the X, Y, and Z slide positions and rotation positions of the corrected candidate teaching point to obtain the position and orientation in the robot coordinate system. The processor 210 uses the position and orientation in the robot coordinate system to obtain the position and orientation in the camera coordinate system. The processor 210 can calculate the position of the workpiece 800 on the two-dimensional image (i.e., the position on the reference image IMr) from the position in the camera coordinate system by interpolation using multiple points P. The same applies to the corrected orientation of the tool 990 on the reference image IMr. The processor 210 can calculate the orientation of the workpiece 800 in the two-dimensional image (that is, the orientation in the reference image IMr) from the orientation in the camera coordinate system by interpolation using multiple points P.

[0119] After completing the correction of the position and orientation, the user operates the end button BT. In response to the operation of the end button BT, the processor 210 proceeds to S258b (FIG. 8).

[0120] In S258b, the processor 210 acquires the position and orientation in the robot coordinate system using the slide positions and rotation positions in the X, Y, and Z directions of the corrected candidate teaching point, and calculates the position and orientation in the camera coordinate system using the position and orientation in the robot coordinate system. The processor 210 generates data for the corrected candidate teaching point using the position and orientation in the camera coordinate system and the slide positions and rotation positions in the X, Y, and Z directions. The processor 210 determines a new target teaching point Pt as the corrected candidate teaching point. Then, the processor 210 stores the data of the target teaching point Pt in the storage device 215 (e.g., the non-volatile storage device 230). The processor 210 also calculates a corrected representative point associated with the new target teaching point Pt (i.e., the corrected candidate teaching point). The calculation method is the same as the calculation method for the corrected representative point cPrm described in S258 of FIG. 4. The processor 210 stores the corrected representative point data in the storage device 215 (eg, the non-volatile storage device 230).

[0121] After S258b, the processor 210 proceeds to S260. The processing of S260 is the same as the processing of S260 in FIG. 4. If the determination result of S260 is Yes, the processor 210 proceeds to S210 and executes processing for the next target teaching point Pt. If the determination result of S260 is No, the processor 210 executes S265-S295 similarly to S265-S295 in FIG. 4. Then, the processor 210 ends the teaching processing of FIGS. 3 and 8. Note that S265 and S280 may be the same as S265 and S280 in FIG. 4, respectively. The presentation method in S270 and S285 may be the same as the presentation method in S215b. The processing of S275 and S290 may be performed in the same manner as the adoption or correction of the candidate teaching point cPt in S225b-S258b.

[0122] As described above, in this embodiment, in S215b and S253b (FIG. 8), the processor 210 presents the candidate teaching point cPt, which is a candidate for the target teaching point Pt, to the user (S215b and S253b as a whole are an example of a presentation process for presenting the candidate teaching point to the user). Specifically, in S215b and S253b, as shown in FIG. 9, the processor 210 presents the candidate teaching point cPt to the user by displaying a reference image IMr, which is an image including an image of the workpiece 800 and an image of the tool 990 positioned at the candidate teaching point cPt, on the display unit 240. Therefore, the user can easily confirm the candidate teaching point cPt by observing the reference image IMr. Furthermore, in S253b, the processor 210 changes the position and orientation of the tool 990 in the displayed reference image IMr according to the instruction. In this way, the processor 210 changes the posture (specifically, the position and orientation) of the tool 990 in accordance with the posture change instruction. Therefore, the user can easily change the posture of the tool 990 while observing the reference image IMr. Note that an image (here, a large double circle) indicating the candidate teaching point cPt, separate from the image of the tool 990, may be omitted from the reference image IMr.

[0123] In this embodiment, as described with reference to FIG. 2, the workpiece 800 and the tool 990 are supported by the support device 930. The support device 930 can change the orientation of the tool 990 with respect to the workpiece 800 by combining movement by the first support device 910 and rotation by the second support device 920. That is, the tool 990 can rotate relatively with respect to the workpiece 800. The rotation axis AXi represented by the reference image IMr ( FIG. 9 ) is the axis of rotation of the tool 990 relative to the workpiece 800. In this way, the support device 930 is configured to change the orientation of the tool with respect to the workpiece 800 by rotating the tool 990 relatively around the rotation axis AXi set with respect to the workpiece 800. The reference image IMr includes an image showing the rotation axis AXi. Therefore, by observing the reference image IMr, the user can easily understand how the orientation of the tool 990 can be changed.

[0124] 3 and 4 except that the presentation and correction of the candidate teaching point cPt are performed on the display unit 240. For example, S210, S215b, S225b, S230b, S248, S253b, and S258b in FIG. 8 correspond to S210, S215, S225, S230, S248, S253, and S258 in FIG. 4, respectively. Therefore, this embodiment can provide the same various advantages as those provided by the embodiment in FIGS. 3 and 4.

[0125] C. Third Example: The portion of the tool 990 that should contact the workpiece 800 may be various portions of the tool 990. For example, instead of the tip 990t of the tool 990, the side of the tool 990 may contact the workpiece 800. In this case, the target position and target orientation represented by the target teaching point Pt are preferably such that the side of the tool 990 faces the representative point Pr. FIG. 10 is a diagram showing an example of a candidate teaching point cPt that represents such a position and orientation. There are two differences from the candidate teaching point cPt in FIG. 7(B). The first difference is that the movement vector MV2 indicating the movement from the representative point cPr to the candidate teaching point cPt includes a component in the direction perpendicular to the normal vector vnc in addition to a component in the direction of the normal vector vnc at the representative point cPr. The magnitude dp of this perpendicular component is called the shift distance dp. The second difference is that the orientation of the tool 990 is opposite to the direction of this perpendicular component. As a result, the side surface 990s of the tool 990 faces the representative point cPr. That is, the side surface 990s of the tool 990 is located in the direction of the normal vector vnc of the representative point cPr. Such a candidate teaching point cPt is applicable to each of the above-described embodiments (e.g., S210 (FIGS. 4 and 8)). The processor 210 may determine the candidate teaching point cPt described in FIG. 10 in S210 (more specifically, S325 and S340 in FIG. 6).

[0126] As described above, in this embodiment, the next candidate determination process S210 (FIGS. 4 and 8) includes S325 (FIG. 6), as in the above-described embodiments. In S325, the processor 210 determines the next candidate teaching point cPt (FIG. 10) to be presented as a specific teaching point located at a position obtained by moving the representative point cPr in the second direction (here, the direction of the movement vector MV2). The representative point cPr is a point acquired by downsampling the point cloud measured by the camera 700, as described in S160 (FIG. 3). The point P included in the downsampled point cloud indicates a point on the outer surface 800o of the workpiece 800 and represents a measured value in three-dimensional coordinates. In this way, the representative point cPr is an example of a second point, which is a point on the outer surface 800o of the workpiece 800 estimated using the measured value in three-dimensional coordinates. 10, the direction of the movement vector MV2 includes a component in the normal direction of the outer surface 800o at the representative point cPr and a component perpendicular to the normal direction (i.e., a component in a direction parallel to the outer surface 800o at the representative point cPr). A specific teaching point indicates the posture (here, the position and orientation) of the tool 990 in a state where the side surface 990s, rather than the tip 990t, of the tool 990 faces the representative point cPr. In this way, the processor 210 can determine that the next candidate teaching point cPt to be presented is a teaching point where the side surface 990s of the tool 990 is positioned near the representative point cPr of the workpiece 800.

[0127] 10, the side surface 990s of the tool 990 is disposed at a position where it intersects with a straight line Ln that passes through the representative point cPr and extends in the direction of the normal vector vnc. Alternatively, the position and orientation of the tool 990 represented by the candidate teaching point cPt may be configured so that the side surface 990s is disposed at a position where it does not intersect with the straight line Ln. For example, the orientation of the tool 990 may be perpendicular to the normal vector vnc and inclined with respect to a direction corresponding to the shift distance dp of the movement vector MV2. In this case, too, if the part of the outer machining surface of the tool 990 (e.g., the portion where the blade is provided) closest to the representative point cPr is the side surface 990s rather than the tip 990t, it can be said that the side surface 990s faces the representative point cPr.

[0128] D. Fourth Example: In each of the above embodiments, the size of the voxels VX is not limited to the size shown in FIG. 5(C) and may be various sizes. FIG. 11(A) is a diagram showing an example of the arrangement of multiple voxels VX when the size of the voxels VX is smaller than the size of the voxels VX in FIG. 5(C). Like FIG. 5(C), FIG. 11(A) shows a workpiece 800 and multiple voxels VX. Among the multiple voxels VX, the diagram shows multiple voxels VX that include a portion of the target region 800A. In the example of FIG. 5(C), one side E1 is represented by multiple voxels VX that form one line. The same applies to sides E2 and E3. In the example of FIG. 11(A), one side E1 is represented by multiple voxels VX that form multiple lines. The same applies to sides E2 and E3.

[0129] FIG. 11(B) is a diagram showing an example of multiple representative points Pr. The figure shows a workpiece 800 and multiple representative points Pr corresponding to the multiple voxels VX in FIG. 11(A). The multiple representative points Pr are arranged along the target region 800A. Compared to the example in FIG. 5(E), the density of the representative points Pr is higher. In this case, too, the processor 210 can determine multiple target teaching points Pt by the teaching process of each of the above embodiments.

[0130] Although not shown in the figure, there are cases where some of the representative points Pr among the representative points Pr cannot be associated with the target teaching point Pt. The proportion of the representative points Pr that cannot be associated with the target teaching point Pt tends to increase as the density of the representative points Pr increases (i.e., as the size of the voxel VX decreases).

[0131] E. Variations: (1) The method of correcting the position of the candidate teaching point cPt is not limited to the method described in FIGS. 4 and 8, and various other methods may be used. For example, the processor 210 may change the position by having the user edit the coordinate values ​​of the position. The coordinate values ​​of the position may be expressed in, for example, the camera coordinate system or the robot coordinate system. The processor 210 may also change the position by having the user edit the slide positions of X, Y, and Z.

[0132] The method of correcting the orientation of the candidate teaching point cPt is not limited to the methods described with reference to FIGS. 4 and 8 , and various other methods may be used. For example, the user may change the orientation of the tool 990 by moving the tool 990 in the reference image IMr ( FIG. 9 ) via the operation unit 250. For example, if the operation unit 250 includes a mouse, the user may change the orientation of the tool 990 by dragging the image of the tool 990 in the reference image IMr using the mouse. If the mouse has a wheel, the orientation of the tool 990 may be changed by rotating the wheel. In either case, the processor 210 calculates the corrected orientation in response to the operation of the operation unit 250. The processor 210 may also change the orientation by having the user edit a numerical value representing the orientation. The numerical value representing the orientation may include, for example, a rotation position (i.e., an angle) around the rotation axis AXi ( FIG. 9 ) or a combination of the X, Y, and Z slide positions and the rotation position by the second support device 920.

[0133] The process of correcting the candidate teaching point cPt may be omitted. For example, in the process of Fig. 4 and Fig. 8, after S215 and S215b, the process may proceed to S248.

[0134] (2) The candidate teaching point cPt may be various other teaching points instead of the teaching points described in Fig. 7(B). For example, the movement direction from the representative point cPr to the candidate teaching point cPt may include other components in addition to the component in the normal direction of the outer surface 800o at the representative point cPr.

[0135] (3) The specific range referenced for determining the next candidate teaching point cPt is not limited to the specific range VRs shown in FIG. 7A and may have various configurations. For example, the calculation method of the three-dimensional vector Vs used to determine the specific range VRs may be various methods based on multiple target teaching points Pt, including the latest target teaching point Ptb. The three-dimensional vector Vs may be calculated based on the latest N (N is an integer greater than or equal to 2) target teaching points Pt. The three-dimensional vector Vs preferably indicates a representative direction of position change when sequentially tracing N representative points Pr corresponding to the N target teaching points Pt. For example, the path sequentially tracing the N representative points Pr is represented by N-1 displacement vectors. The three-dimensional vector Vs may be an average vector of the N-1 displacement vectors. The target teaching point Pt is located near the corresponding representative point Pr. Therefore, the three-dimensional vector Vs may be calculated according to the position of the target teaching point Pt, instead of the representative point Pr. The direction of the three-dimensional vector Vs may be parallel to a line that approximates the arrangement of the N target teaching points Pt or the arrangement of the N representative points Pr.

[0136] The point indicating the position reference of the specific range VRs is not limited to a point on the outer surface 800o associated with the latest target teaching point Ptb (here, the representative point Prb), but may be any of various points associated with the latest target teaching point Ptb. For example, a range extending from the latest target teaching point Ptb in the direction of the three-dimensional vector Vs may be used as the specific range VRs.

[0137] The shape of the specific area is not limited to a cylinder, but may be any shape extending in the direction of the three-dimensional vector Vs (for example, a square prism).

[0138] (4) The designated area (e.g., target area 800A (FIG. 5(A))) referenced to determine the candidate teaching point cPt is not limited to an area colored in a specific color, but may be various areas. For example, an area surrounded by lines of a specific color may be referenced. Furthermore, the designated area is not limited to an area including an edge (e.g., edges E1, E2, E3), but may be located in the center of a plane. Note that the designation of the area may be omitted. The processor 210 may determine the candidate teaching point cPt to be a teaching point associated with a specific portion of the outer surface 800o (e.g., a portion forming an edge, a portion forming a vertex, etc.).

[0139] (5) The process for determining the next candidate teaching point cPt may be various other processes instead of the process described in FIG. 6. For example, the determination of the candidate teaching point cPt by referring to a specific range (for example, the specific range VRs (FIG. 7(A))) may be omitted. Specifically, S310-S325 in FIG. 6 may be omitted, and the position of the candidate teaching point cPt may be determined by the processes of S330, S335, S340, and S345. Alternatively, S330-S340 may be omitted, and if the determination result of S320 is No, the process may proceed to S345.

[0140] (6) The method of presenting the candidate teaching point cPt to the user is not limited to the method described in FIGS. 4 and 8, and various other methods may be used. For example, the image of the rotation axis AXi may be omitted from the reference image IMr (FIG. 9). The image of the representative point Pr may be omitted from the reference image IMr. In S215 of FIG. 4, the processor 210 may further present the candidate teaching point cPt to the user using the display unit 240, similar to S215b of FIG. 8. In S270 and S285, the processor 210 may actually move the tool 990 to the teaching point and present the teaching point to the user using the display unit 240.

[0141] (7) The teaching process may be various other processes instead of the processes described in Figures 3, 4, 8, etc. For example, the process of determining the approach point PtS (S265-S275) in Figures 4 and 8 may be omitted. The process of determining the escape point PtE (S280-S290) in Figures 4 and 8 may be omitted. Furthermore, the density of the multiple points in the point cloud acquired by the camera 700 may be various. If the density of the multiple points is low, downsampling (Figure 3, S160) may be omitted.

[0142] (8) The configuration of the processing device may be various other configurations instead of the configuration of the processing device 900 described in FIG. 2 . For example, the support device may include an articulated robot arm (e.g., a six-axis robot arm). The tool 990 may be supported by this robot arm. The power source of the processing device may be various other power sources (e.g., hydraulic, pneumatic, etc.) instead of an electric motor. A control device (e.g., control device 980) of the processing device may be configured to receive a position change instruction and an orientation change instruction in a coordinate system fixed to the workpiece 800 (referred to as a workpiece coordinate system). In this case, a reference object (e.g., an AR marker) that defines the workpiece coordinate system may be attached to a support device (e.g., table 920t) that supports the workpiece 800. Then, the processor 210 may determine the correspondence between the camera coordinate system and the workpiece coordinate system using information about the reference object read by the camera 700. The processor 210 may generate data of the teaching point expressed in the workpiece coordinate system. The machining device may be configured to change the position of the tool relative to the workpiece without changing the orientation of the tool relative to the workpiece, in which case the processor 210 may generate data for the teaching points that represent the position without representing the orientation.

[0143] (9) The processing of the workpiece by the tool is not limited to deburring, and may be various processing (e.g., welding, polishing, sealing, etc.). The tool may be various tools suitable for the processing (e.g., welding torch, polishing spindle, application gun, etc.).

[0144] (9) Data processing device 200 in Fig. 1 may be a device of a type different from a personal computer (e.g., a digital camera, a scanner, a smartphone, or a tablet computer). Furthermore, multiple devices (e.g., computers) that can communicate with each other via a network may share some of the data processing functions of the data processing device and collectively provide the data processing functions (a system including these devices corresponds to a data processing device).

[0145] In each of the above embodiments, a part of the configuration realized by hardware may be replaced by software, and conversely, a part or all of the configuration realized by software may be replaced by hardware. For example, the process of determining the next candidate teaching point cPt (e.g., the process of FIG. 6) may be executed by a dedicated hardware circuit such as an Application Specific Integrated Circuit (ASIC).

[0146] Furthermore, when some or all of the functions of the present disclosure are realized by a computer program, the program can be provided in a form stored on a computer-readable recording medium (e.g., a non-transitory recording medium). The program can be used in a state stored on the same or a different recording medium (computer-readable recording medium) from when it was provided. The "computer-readable recording medium" is not limited to portable recording media such as memory cards and CD-ROMs, but can also include internal storage devices within a computer, such as various ROMs, and external storage devices connected to a computer, such as a hard disk drive.

[0147] The above-described examples and modifications can be combined as appropriate. The above-described examples and modifications are provided to facilitate understanding of the present disclosure and are not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit thereof, and the present invention includes equivalents thereof. [Explanation of symbols]

[0148] 200...data processing device, 210...processor, 215...storage device, 220...volatile storage device, 230...non-volatile storage device, 231...program, 240...display unit, 250...operation unit, 270...communication interface, 600...controller, 700...camera, 800...workpiece, 800A...target area, 800o...outer surface, 800op...specific part, 900...processing device, 930...support device, 910...first support device, 910x, 910y, 910z...arm, 920...second support device, 920s...support wall, 920t...table, 980...control device, 990...tool, 990ax...central axis, 990s...side, 990t...tip, 990u...tool unit, AXi,Da...rotation axis

Claims

1. A program, a function of executing a presentation process of presenting to a user candidate teaching points that are candidates for target teaching points that form a path for machining an object by a tool; a function of executing a teaching point determination process for determining a new target teaching point in accordance with an instruction from the user, the teaching point determination process including a process of determining the new target teaching point as the presented candidate teaching point when the instruction from the user is an adoption instruction indicating adoption of the presented candidate teaching point; a function of executing a next candidate determination process for determining a candidate teaching point to be presented next based on one or more target teaching points including the new target teaching point; A function of determining a plurality of target teaching points by executing a determination process, the determination process including a process of adding new target teaching points one by one to the plurality of target teaching points by repeating the presentation process, the teaching point determination process, and the next candidate determination process; A program that makes the computer realize the above.

2. 2. The program according to claim 1, the teaching point determination process includes a process of determining the new target teaching point as a corrected teaching point when the user's instruction is a correction instruction indicating correction of the presented candidate teaching point. program.

3. 3. The program according to claim 1 or 2, the object has an exterior surface that includes a designated area; the next candidate determination process includes a process of determining the next candidate teaching point to be presented as a teaching point associated with the specified area, program.

4. 3. The program according to claim 1 or 2, the object has an outer surface; The next candidate determination process includes: A process of calculating a vector based on a plurality of target teaching points including the latest target teaching point; a process of determining the next candidate teaching point to be presented as a teaching point associated with a specific portion that is a part of the outer surface of the object, the specific portion being a portion within a specific three-dimensional range extending in the direction of the vector from a reference point that is a point on the outer surface that is associated with the latest target teaching point; Including, the program.

5. 3. The program according to claim 1 or 2, The presentation process includes: a process of presenting the candidate teaching points to the user by displaying on a display device a reference image that includes an image of the object and an image of the tool located at the candidate teaching points; a process of changing the orientation of the tool in the reference image being displayed in accordance with an orientation change instruction; Including, the program.

6. 6. The program according to claim 5, the object and the tool are supported by a support device; the support device is configured to change the orientation of the tool with respect to the object by rotating the tool relatively around a rotation axis set with respect to the object; the reference image includes an image showing the rotation axis; program.

7. 3. The program according to claim 1 or 2, The next candidate determination process includes a process of determining the next candidate teaching point to be presented as a teaching point located at a position obtained by moving a first point in a first direction, the first point being a point on an outer surface of the object estimated using measurement values ​​of three-dimensional coordinates, and the first direction including a component of a normal direction at the first point on the outer surface. program.

8. 3. The program according to claim 1 or 2, further comprising: a function of determining a candidate approach point that is a candidate for an approach point that is a teaching point immediately before a first target teaching point among the plurality of target teaching points; a function of presenting the candidate approach points to the user; a function of determining the approach point to be one of the candidate approach points or a teaching point designated by the user in accordance with an instruction from the user; a function of determining a candidate evacuation point that is a candidate for the evacuation point, which is a teaching point immediately after the last target teaching point among the plurality of target teaching points; a function of presenting the candidate evacuation points to the user; a function of determining the evacuation point as the candidate evacuation point or a teaching point designated by the user in accordance with an instruction from the user; A program that enables a computer to achieve this.

9. 3. The program according to claim 1 or 2, The next candidate determination process includes a process of determining the next candidate teaching point to be presented as a specific teaching point located at a position obtained by moving a second point in a second direction, the second point being a point on the outer surface of the object estimated using measurement values ​​of three-dimensional coordinates, the second direction including a component of a normal direction at the second point on the outer surface and a component of a direction parallel to the outer surface at the second point, and the specific teaching point indicating the attitude of the tool in a state where the side of the tool, not the tip, faces the second point. program.

10. 1. A data processing device, comprising: a presentation unit that executes a presentation process to present to a user candidate teaching points that are candidates for target teaching points that form a path for machining an object by a tool; an instructed teaching point determination unit that executes a teaching point determination process to determine a new target teaching point in accordance with an instruction from the user, the teaching point determination process including a process of determining the new target teaching point as the presented candidate teaching point when the instruction from the user is an adoption instruction indicating adoption of the presented candidate teaching point; a next candidate determination unit that executes a next candidate determination process to determine a candidate teaching point to be presented next based on one or more target teaching points including the new target teaching point; a teaching point determination unit that determines a plurality of target teaching points by executing a determination process, the determination process including a process of adding new target teaching points one by one to the plurality of target teaching points by repeating the presentation process, the teaching point determination process, and the next candidate determination process; A data processing device comprising:

11. 1. A data processing method comprising: Executing a presentation process to present to a user candidate teaching points that are candidates for target teaching points that form a path for machining the object by the tool; Executing a teaching point determination process for determining a new target teaching point in accordance with an instruction from the user, the teaching point determination process including a process for determining the new target teaching point as the presented candidate teaching point when the instruction from the user is an adoption instruction indicating adoption of the presented candidate teaching point; Executing a next candidate determination process for determining a next candidate teaching point to be presented based on one or more target teaching points including the new target teaching point; Equipped with The data processing method further comprises: determining a plurality of target teaching points by executing a determination process, the determination process including a process of adding new target teaching points one by one to the plurality of target teaching points by repeating the presentation process, the teaching point determination process, and the next candidate determination process; Data processing methods.

Images