Processing machine, control device for processing machine, and control method for processing machine

Abstract

In the present invention, a target identification unit identifies the target shape of a workpiece on the basis of a first depth image obtained by imaging a sample representing the target shape by a depth camera. A position measurement unit calculates the three-dimensional position of the surface of the workpiece on the basis of a second depth image obtained by imaging the workpiece by the depth camera. A positioning unit calculates the three-dimensional position of the target shape of the workpiece when the workpiece and the target shape are superimposed, on the basis of three-dimensional data representing the target shape and the three-dimensional position of the workpiece. A depth-of-cut determination unit determines a depth of cut at each of multiple points on the surface of the workpiece on the basis of a difference, along the line of sight, between the three-dimensional position of the surface of the workpiece viewed from a predetermined viewpoint and the three-dimensional position of the surface of the target shape. A tool control unit moves a tool on the basis of the determined depth of cut.

Description

Processing machine, processing machine control device, and processing machine control method

[0001] This disclosure relates to a processing machine, a control device for a processing machine, and a control method for a processing machine. This application claims priority to Japanese Patent Application No. 2023-221709, filed in Japan on December 27, 2023, the contents of which are incorporated herein by reference.

[0002] The control device of the processing machine reads the processing program used to process the workpiece and executes the program to drive the processing machine to cut the workpiece into a desired shape. In contrast, Patent Document 1 discloses a technology that allows a workpiece to be processed without preparing a processing program.

[0003] Japanese Patent Application Laid-Open No. 2023-113305

[0004] According to the control device disclosed in Patent Document 1, it is necessary to prepare three-dimensional data representing a target shape in order to control a processing machine. However, there are cases where a sample of the target shape is available but the three-dimensional data is not. An object of the present disclosure is to provide a processing machine, a control device for the processing machine, and a control method for the processing machine that can process a workpiece according to the target shape represented by the sample based on an actual sample.

[0005] According to one aspect of the present invention, a processing machine includes a tool for machining a workpiece, a depth camera installed so that the workpiece is within its imaging range and for capturing images of the depth of the subject, a target identification unit for identifying the target shape based on a first depth image captured by the depth camera of a sample representing the target shape of the workpiece, a position measurement unit for calculating the three-dimensional position of the surface of the workpiece based on a second depth image captured by the depth camera of the workpiece, an alignment unit for calculating the three-dimensional position of the target shape when the workpiece and the target shape are superimposed based on the target shape and the three-dimensional position of the workpiece, a cutting depth determination unit for determining the cutting depth at each of multiple points on the surface of the workpiece based on the difference in viewing direction between the three-dimensional position of the surface of the workpiece when the workpiece is viewed from a predetermined viewpoint and the three-dimensional position of the surface of the target shape, and a tool control unit for moving the tool based on the determined cutting depth.

[0006] According to the above aspect, it is possible to machine a workpiece based on an actual sample in accordance with a target shape represented by the sample.

[0007] FIG. 1 is a perspective view of a processing machine according to a first embodiment; FIG. 2 is a perspective view showing the configuration of a robot arm according to the first embodiment; FIG. 3 is a schematic block diagram showing the configuration of a control device according to the first embodiment; FIG. 4 is an example of a display screen according to the first embodiment; FIG. 5 is a flowchart (part 1) showing a method of controlling a processing machine by the control device according to the first embodiment; FIG. 6 is a flowchart (part 2) showing a method of controlling a processing machine by the control device according to the first embodiment; FIG. 7 is a flowchart showing a method of controlling a processing machine by the control device according to a second embodiment; and FIG. 8 is a schematic block diagram showing the configuration of a computer according to at least one embodiment.

[0008] First Embodiment Configuration of Processing Machine 1 Hereinafter, embodiments will be described in detail with reference to the drawings. FIG. 1 is a perspective view of a processing machine according to a first embodiment. The processing machine 1 includes a table 10, a jig 20, a plurality of stereo cameras 30, a plurality of projectors 31, a robot arm 40, a tool 60, and a control device 70. In other embodiments, the processing machine 1 may be, for example, a machining center. The processing machine 1 according to the first embodiment recognizes the shape of a sample S prepared by a user and cuts out the shape of the sample S from a workpiece W. The sample S represents a target shape for cutting.

[0009] The table 10 is a base of the processing machine 1. The jig 20 is provided on the table 10. The jig 20 supports the workpiece W by clamping it therebetween.

[0010] The stereo camera 30 captures depth images of a subject included in its imaging range. The stereo camera 30 is an example of a depth camera. The stereo camera 30 is installed so that its imaging range includes the area where the workpiece W is installed. The stereo camera 30 is installed, for example, on the inner wall of the processing machine 1 so as to overlook the table 10. The stereo camera 30 is preferably fixed with a jig made of a material that is resistant to thermal expansion so that its relative position with respect to the table 10 does not change. Multiple stereo cameras 30 are arranged to cover each other's blind spots. In the example shown in FIG. 1, the processing machine 1 is equipped with four stereo cameras 30, which can capture depth images from four directions. Note that in other embodiments, the number of stereo cameras 30 may be one or more. Furthermore, depth images may be measured from two or one direction instead of four directions. The stereo cameras 30 are attached to the four corners of the ceiling of a substantially rectangular parallelepiped housing that covers the processing machine 1. Note that the stereo cameras 30 only need to be fixed to the processing machine 1; they may be fixed not only to the housing but also to, for example, four pillars.

[0011] The projector 31 projects a random dot pattern into the imaging range of the stereo camera 30. The random dot pattern is a pattern in which dots of light are irregularly arranged. The random dot pattern is an example of a non-periodic pattern. A projector according to another embodiment may project other non-periodic patterns, such as a pattern in which multiple curves are irregularly arranged. The multiple projectors 31 are provided, for example, near each of the multiple stereo cameras 30. The multiple projectors 31 are arranged so as to compensate for each other's blind spots. By capturing an image of the subject onto which the random dot pattern is projected, the stereo camera 30 can maintain the accuracy of stereo matching even when the subject has a flat surface or a periodic shape or pattern.

[0012] FIG. 2 is a perspective view showing the configuration of a robot arm 40 according to the first embodiment. The robot arm 40 movably supports a tool 60. The tool 60 is supported at the tip of the robot arm 40. The tool 60 cuts a workpiece W. In another embodiment, the tool 60 may grind the workpiece W. The robot arm 40 may be, for example, a six-axis vertical joint robot. The robot arm 40 includes a base 41, a first arm 42, a second arm 43, a third arm 44, a fourth arm 45, a fifth arm 46, and a spindle 47. The base 41 is fixed to the table 10 so as to be rotatable about a first axis O1 extending in the vertical direction. The first arm 42 is connected to the table 10 so as to be rotatable about a second axis O2 perpendicular to the first axis O1. The second arm 43 is connected to the first arm 42 so as to be rotatable about a third axis O3 parallel to the second axis O2. The third arm 44 is connected to the table 10 so as to be rotatable about a fourth axis O4 parallel to the third axis O3. The fourth arm 45 is connected to the third arm 44 so as to be rotatable about a fifth axis O5 extending in the axial direction of the third arm 44. The fifth arm 46 is connected to the fourth arm 45 so as to be rotatable about a sixth axis O6 perpendicular to the fifth axis O5. The spindle 47 is provided at the tip of the fifth arm 46 and rotatably supports a tool 60.

[0013] The robot arm 40 includes a first motor 51, a second motor 52, a third motor 53, a fourth motor 54, a fifth motor 55, a sixth motor 56, and a seventh motor 57. The first motor 51 rotates the base 41. The second motor 52 rotates the first arm 42 relative to the base 41. The third motor 53 rotates the second arm 43 relative to the first arm 42. The fourth motor 54 rotates the third arm 44 relative to the second arm 43. The fifth motor 55 rotates the fourth arm 45 relative to the third arm 44. The sixth motor 56 rotates the fifth arm 46 relative to the fourth arm 45. The seventh motor 57 rotates the spindle 47. Each motor is provided with an encoder (not shown) that measures the rotation angle of each arm.

[0014] The robot arm 40 includes a driver 58. The driver 58 drives each motor of the robot arm 40 in accordance with a control instruction. The driver 58 calculates the position and orientation of the tip of the spindle 47 based on the rotation angle measured by the encoder. The driver 58 specifies the position and orientation of the spindle 47 in a base coordinate system and a tool coordinate system. The base coordinate system has its origin at the center of the bottom surface of the base 41 and is represented by a Zb axis extending in the vertical direction and Xb and Yb axes perpendicular to the Zb axis. The tool coordinate system has its origin at the tip of the spindle 47 and is represented by a Zt axis extending in the axial direction of the spindle 47 and Xt and Yt axes perpendicular to the Zt axis. When the driver 58 receives a control instruction indicating the coordinate axis and the amount of translation or rotation, the driver 58 calculates the angle of each motor to move the spindle 47 by the amount indicated by the control instruction along the coordinate axis indicated by the control instruction, and controls each motor. Furthermore, when the driver 58 receives a control instruction indicating a coordinate system and a position and attitude in that coordinate system, it calculates the angle of each motor to move the spindle 47 to the position indicated by the control instruction, and controls each motor.

[0015] The control device 70 controls the robot arm 40 based on the depth image captured by the stereo camera 30 and the position of the tip of the robot arm 40 .

[0016] 3 is a schematic block diagram showing the configuration of the control device 70 according to the first embodiment. The control device 70 includes a data acquisition unit 71, a storage unit 72, a target identification unit 73, a position measurement unit 74, an alignment unit 75, a display control unit 76, a difference calculation unit 77, a cutting amount determination unit 78, a path generation unit 79, a tool control unit 80, and a correction unit 81.

[0017] The data acquisition unit 71 acquires depth images from the stereo camera 30 and acquires measurement values ​​of the angles of each arm from the robot arm 40. The depth image is obtained by associating each pixel constituting an image captured by one of the cameras, which is the main camera of the stereo camera 30, with a depth indicating the distance from that camera. The data acquisition unit 71 acquires, from the stereo camera 30, a first depth image in which the sample S placed on the workpiece W is captured and a second depth image in which the workpiece W is captured.

[0018] The storage unit 72 stores tool data, which is three-dimensional data indicating the shape of the tool 60, and matching data, which is three-dimensional data indicating the shape of the workpiece W before machining. The tool data and matching data may be, for example, CAD data. The storage unit 72 also stores the position and orientation of the stereo camera 30 in the base coordinate system.

[0019] The target identification unit 73 identifies the three-dimensional shape of the sample S based on the first depth image acquired from the stereo camera 30 and generates target data, which is three-dimensional data representing the target shape of the workpiece W. Specifically, the target identification unit 73 according to the first embodiment identifies the three-dimensional shape of the sample S in the following procedure. First, the target identification unit 73 generates point cloud data in the base coordinate system based on the first depth image generated by at least one stereo camera 30 and the position and orientation of the stereo camera 30 in the base coordinate system. The point cloud data represents the shape of the workpiece W and the sample S integrated together. Next, the target identification unit 73 calculates the difference between the height of the shape represented by the point cloud data and the height of the workpiece W represented by the matching data stored in the memory unit 72. The target identification unit 73 removes the bottom of the shape represented by the point cloud data by the calculated height, and sets the result as target data. This allows the target identification unit 73 to generate target data that minimizes the amount of cutting from the workpiece W.

[0020] The position measurement unit 74 identifies the three-dimensional position of the workpiece W based on the depth images acquired from the stereo camera 30. Specifically, the position measurement unit 74 according to the first embodiment identifies the three-dimensional position of the workpiece W in the following procedure. First, the position measurement unit 74 generates point cloud data in a base coordinate system based on the depth images generated by at least one stereo camera 30 and the position and orientation of the stereo camera 30 in the base coordinate system. Next, the position measurement unit 74 matches the point cloud data with the matching data stored in the storage unit 72. The position measurement unit 74 can perform matching using, for example, an ICP algorithm. As a result, the position measurement unit 74 identifies the position and orientation of the matching data in the base coordinate system as the three-dimensional position of the workpiece W.

[0021] The alignment unit 75 places the target data generated by the target identification unit 73 at the three-dimensional position of the workpiece W measured by the position measurement unit 74. That is, the alignment unit 75 determines the position and orientation in the base coordinate system of the target shape of the workpiece W represented by the target data. The alignment unit 75 generates a depth image corresponding to the stereo camera 30 from the aligned target shape. That is, the alignment unit 75 generates a depth image when the target shape of the workpiece W is projected from the viewpoint of the stereo camera 30 based on the position and orientation in the base coordinate system of the stereo camera 30 stored in the memory unit 72 and the position and orientation in the base coordinate system of the target shape. Hereinafter, the depth image generated by the alignment unit 75 is referred to as a target depth image. The alignment unit 75 is an example of a target depth calculation unit that calculates the depth of the surface of the target shape relative to the stereo camera 30 based on the three-dimensional position of the target shape.

[0022] The display control unit 76 renders a target image P1 representing the target shape of the workpiece W as seen from one of the stereo cameras 30, which serves as the main camera, for each stereo camera 30 based on the target depth image generated by the alignment unit 75. The target image P1 may be, for example, a line drawing obtained by outline rendering of a three-dimensional model or a translucent image. The display control unit 76 displays a display screen on the display, in which the image captured by the stereo camera 30 (captured image P0) and the target image P1 corresponding to the stereo camera 30 are superimposed. FIG. 4 is an example of a display screen according to the first embodiment. Note that the display control unit 76 does not necessarily need to display information such as alignment on the display screen. As shown in FIG. 4, the target image P1 is displayed so as to be included in the workpiece W captured in the captured image P0. The captured image P0 may be a composite of images captured by the four stereo cameras 30.

[0023] The difference calculation unit 77 calculates the depth difference for each pixel between the second depth image acquired by the stereo camera 30 and the target depth image generated by the alignment unit 75. The difference calculation unit 77 converts the coordinate systems of the second depth image and the target depth image into a base coordinate system, thereby aligning one axis of the coordinate system representing the second depth image with the direction in which the tool 60 faces (vertically downward direction), and then calculates the depth difference.

[0024] The cutting depth determination unit 78 determines the cutting depth at a position of the workpiece W corresponding to each pixel based on the depth difference of each pixel and the maximum cutting depth of the tool 60. A pixel of the workpiece W represents a point on the surface of the workpiece W. Specifically, the cutting depth determination unit 78 determines the cutting depth using the following procedure. First, the cutting depth determination unit 78 identifies the maximum depth difference calculated by the difference calculation unit 77. The cutting depth determination unit 78 determines the expected number of scans by adding 1 to the integer part of the value obtained by dividing the maximum depth difference by the maximum cutting depth. Note that in another embodiment, the expected number of scans may be determined using a predetermined cutting depth smaller than the maximum cutting depth set in advance by the operator instead of the maximum cutting depth. The expected number of scans is the number of scans required to complete cutting if the workpiece W is cut as planned. Note that the actual number of scans does not necessarily match the expected number of scans due to influences such as control errors of the robot arm 40 and deflection of the tool 60. The incision amount determination unit 78 determines the incision amount at a position corresponding to a pixel whose depth difference is equal to or greater than the maximum incision amount as the maximum incision amount. The incision amount determination unit 78 determines the incision amount at a position corresponding to a pixel whose depth difference is less than the maximum incision amount as the amount obtained by dividing the depth difference by the number of scans. As a result, for positions corresponding to pixels whose depth difference is less than the maximum incision amount, cutting with small incision amounts can be repeatedly performed, thereby achieving highly accurate cutting. In other embodiments, the incision amount determination unit 78 may determine the incision amount at a position corresponding to each pixel using other methods. For example, the incision amount determination unit 78 according to other embodiments may determine the incision amount as the amount obtained by dividing the depth difference by the number of scans, regardless of the magnitude of the depth difference. Furthermore, the incision amount determination unit 78 according to other embodiments may determine the incision amount at a position corresponding to a pixel whose depth difference is less than the maximum incision amount as the incision amount equivalent to the depth difference. Note that the incision amount determination unit 78 may thin out pixels in the second depth image and calculate the incision amount for only the remaining pixels. In this case, the cut-in amount determination unit 78 may calculate the cut-in amount by interpolating the thinned-out pixels using a moving average.

[0025] The path generating unit 79 determines a path, which is a movement route of the tip of the tool 60, based on the second depth image acquired from the stereo camera 30 and the cutting amount determined by the cutting amount determining unit 78. The path generating unit 79 determines the position of the tip of the tool 60 during cutting by adding the cutting amount determined by the cutting amount determining unit 78 to the depth of each pixel of the second depth image, and determines the path of the tool 60 according to a predetermined path planning algorithm. Examples of path planning algorithms include the Dijkstra algorithm, the A* method, the PRM method, the RRT method, and the RRT* method. The path generating unit 79 may generate a smooth path by interpolating the position of the tip of the tool 60 using a NURBS curve.

[0026] The tool control unit 80 generates a movement instruction to be output to the driver 58 of the robot arm 40 so as to move the tool according to the path generated by the path generation unit 79. The tool control unit 80 first generates a movement instruction to orient the attitude of the tool 60 (rotation angles around each axis in the base coordinate system) vertically downward, and transmits the movement instruction to the driver 58. Thereafter, the tool control unit 80 generates a movement instruction based on the path generated by the path generation unit 79, and transmits the movement instruction to the driver 58. At this time, the tool control unit 80 corrects the position in the Zt axis direction to a position that is shorter than the tip position of the tool 60 indicated by the path by the length of the tool 60.

[0027] After one cutting pass is completed by the tool control unit 80, the correction unit 81 corrects the deviation between the instruction from the tool control unit 80 and the machining position of the robot arm 40. Specifically, the correction unit 81 calculates the deviation amount d of the Xb axis that minimizes the SAD (Sum of Absolute Difference) shown in the following formula (1): ｘ and the deviation amount d in the Yb-axis direction ｙ Explore.

[0028]

[0029] In formula (1), x is the Xb-axis component of the base coordinate system, and y is the Yb-axis component of the base coordinate system. h is the length of the smallest rectangle that contains the path in the Xb-axis direction, and w is the length of the smallest rectangle that contains the path in the Yb-axis direction. p(x, y) is the cutting instruction amount at position (x, y) in the base coordinate system. δ is a constant to prevent division by zero. I(x, y) is the depth difference at position (x, y) between the second depth image captured before machining and the second depth image captured after machining, i.e., the actual cutting amount. The value d that minimizes SAD in formula (1) is ｘ , d ｙ However, since SAD represents a deviation between the instruction of the tool control unit 80 and the machining position of the robot arm 40, the correction unit 81 corrects the control deviation by subtracting the calculated deviation amount from the subsequent movement instruction. Note that, in order to prevent overcutting, the correction unit 81 may add a significantly large penalty value to the SAD when there is a portion where the actual cutting depth is larger than the instructed cutting depth amount.

[0030] <<Control of Processing Machine 1>> Fig. 5 is a flowchart (part 1) showing a method for controlling the processing machine 1 by the control device 70 according to the first embodiment. Fig. 6 is a flowchart (part 2) showing a method for controlling the processing machine 1 by the control device 70 according to the first embodiment. An operator places a workpiece W on the table 10, and then places a sample S on top of the workpiece W. At this time, the operator places the sample S so that it fits inside the workpiece W when viewed from above in a plan view. When the operator inputs an instruction to specify the target shape, the control device 70 starts measuring the sample S.

[0031] First, the data acquisition unit 71 instructs the projector 31 to project a random dot pattern and acquires a depth image from the stereo camera 30 (step S1). This depth image is a first depth image that captures the workpiece W and the sample S. The target identification unit 73 generates point cloud data in the base coordinate system based on the first depth image and the position and orientation of the stereo camera 30 in the base coordinate system stored in the storage unit 72 (step S2). The target identification unit 73 merges the four generated point cloud data (step S3).

[0032] The target identification unit 73 calculates the difference between the height of the shape represented by the point cloud data obtained in step S3 and the height of the workpiece W represented by the matching data stored in the memory unit 72 (step S4). The target identification unit 73 removes the bottom of the shape represented by the point cloud data obtained in step S3 by the height calculated in step S4 to generate target data (step S5). The target identification unit 73 determines whether the target shape represented by the target data generated in step S5 can be contained within the shape of the workpiece W represented by the matching data stored in the memory unit 72 (step S6). If the target shape cannot be contained within the shape of the workpiece W (step S6: NO), the display control unit 76 displays an error message on the display indicating that the sample S protrudes from the workpiece W (step S7). The control device 70 returns to step S1 and waits for an instruction to identify the target shape again. The user adjusts the position of the sample S and inputs an instruction to identify the target shape into the control device 70.

[0033] On the other hand, if the target shape can be contained within the shape of the workpiece W (step S6: YES), the display control unit 76 renders the target data and displays it on the display (step S8). The control device 70 waits for an instruction to specify the target shape or an instruction to start processing (step S9). The user checks the target data displayed on the display, and if there are no problems, removes the sample S from above the workpiece W and inputs an instruction to start processing to the control device 70 (step S9: instruction to start processing). On the other hand, if there are problems with the target data displayed on the display, the user adjusts the position of the sample S and again inputs an instruction to specify the target shape to the control device 70 (step S9: instruction to specify target shape).

[0034] When a processing start command is input, the data acquisition unit 71 instructs the projector 31 to project a random dot pattern and acquires a depth image from the stereo camera 30 (step S10). This depth image is a second depth image in which the workpiece W is captured. Next, the position measurement unit 74 generates point cloud data in the base coordinate system based on the second depth image generated by the stereo camera 30 and the position and orientation of the stereo camera 30 in the base coordinate system stored in the storage unit 72 (step S11). The position measurement unit 74 merges the four generated point cloud data (step S12).

[0035] The position measurement unit 74 identifies the three-dimensional position of the workpiece W by matching the matching data stored in the storage unit 72 with the merged point cloud data (step S13). Next, the alignment unit 75 places the target data generated in step S5 at the three-dimensional position of the workpiece W identified in step S13 in the virtual space (step S14). Next, the alignment unit 75 generates a target depth image corresponding to the stereo camera 30 from the aligned target shape (step S15). The alignment unit 75 records the generated target depth image in the storage unit 72.

[0036] The display control unit 76 renders a target image representing the target shape of the workpiece W as seen from one of the stereo cameras 30 based on the target depth image corresponding to the stereo camera 30 generated in step S15 (step S16). The display control unit 76 records the generated target image in the memory unit 72. The display control unit 76 displays a display screen on the display, in which an image captured by one of the stereo cameras 30 and the target image generated in step S15 are superimposed (step S17). Thereafter, each time the display control unit 76 acquires an image captured by the stereo camera 30 at a constant frame rate, the display control unit 76 updates the display screen by superimposing the image on the target image stored in the memory unit 72, and displays the image on the display. This allows the worker to compare the shape of the workpiece W with the target shape in real time.

[0037] Next, the control device 70 selects the stereo cameras 30 one by one (step S18), and executes the following processes from step S19 to step S31.

[0038] The data acquisition unit 71 acquires a new depth image (second depth image) from the stereo camera 30 selected in step S18 (step S19). The difference calculation unit 77 reads the orientation of the stereo camera 30 selected in step S18 from the storage unit 72 and converts the coordinate systems of the second depth image and the target depth image into a base coordinate system (step S20). That is, the difference calculation unit 77 aligns one axis of the coordinate system representing the second depth image with the direction in which the tool 60 faces (vertically downward). Next, the difference calculation unit 77 calculates the depth difference for each pixel between the second depth image and the target depth image (step S21).

[0039] Next, the cutting amount determination unit 78 determines the expected number of scans by adding 1 to the integer part of the value obtained by dividing the maximum value of the calculated depth difference by the maximum cutting amount of the tool 60 (step S22). The cutting amount determination unit 78 determines the cutting amount at a position corresponding to each pixel of the second depth image based on the depth difference, the maximum cutting amount of the tool 60, and the expected number of scans (step S23). Specifically, the cutting amount determination unit 78 determines the cutting amount at a position corresponding to a pixel whose depth difference is equal to or greater than the maximum cutting amount as the maximum cutting amount. The cutting amount determination unit 78 determines the cutting amount at a position corresponding to a pixel whose depth difference is less than the maximum cutting amount as the amount obtained by dividing the depth difference by the expected number of scans. Alternatively, the cutting amount at a position corresponding to a pixel whose depth difference is less than the maximum cutting amount may be determined as the amount obtained by dividing the depth difference by the maximum cutting amount.

[0040] The path generating unit 79 generates a path for the tool 60 based on the second depth image and the cutting amount determined by the cutting amount determining unit 78 (step S24). If a correction amount is recorded in the memory unit 72, the path generating unit 79 corrects the path based on the correction amount. Next, the tool control unit 80 issues a movement instruction to move the tool according to the path generated in step S24 and outputs the instruction to the driver 58 of the robot arm 40 (step S25). As a result, the robot arm 40 moves the tool according to the path and cuts the workpiece.

[0041] Next, the data acquisition unit 71 acquires a new second depth image from the stereo camera 30 selected in step S18 (step S26). Next, the difference calculation unit 77 calculates the depth difference for each pixel between the second depth image acquired in step S26 and the target depth image (step S27).

[0042] The control device 70 determines whether the depth differences of all pixels are less than the predetermined tolerance (step S28). If the depth difference of at least one pixel is not less than the predetermined tolerance (step S28: NO), the control device 70 determines to continue cutting the surface imaged by the stereo camera 30 selected in step S18.

[0043] If cutting is to continue, the correction unit 81 calculates the actual cutting depth at the position corresponding to each pixel in the second depth image by calculating the depth difference between the second depth image used to generate the path and the newly acquired second depth image (step S29). Next, the correction unit 81 searches for the deviation amount of the machining position using the above-described formula (1) based on the cutting depth indicated by the instruction output to the driver 58 in step S25 and the actual cutting depth (step S30). The correction unit 81 updates the correction amount by adding the obtained deviation amount to the correction amount stored in the memory unit 72 (step S31). Note that if the correction amount is not recorded in the memory unit 72, the correction unit 81 records the obtained deviation amount in the memory unit 72 as the correction amount. The control device 70 then returns the process to step S22 and performs cutting again based on the second depth image acquired in step S26 and the depth difference calculated in step S27.

[0044] If the depth differences of all pixels are less than the predetermined allowable error (step S28: YES), the control device 70 selects the next stereo camera 30 and executes the processes of steps S19 to S31. When the processes of steps S19 to S31 are completed for all stereo cameras 30, the control device 70 determines that the processing of the workpiece W is completed and ends the processing. In other words, the control device 70 ends the processing of the workpiece W when the difference between the target shape and the image viewed from the multiple stereo cameras 30 provided to compensate for blind spots is less than the allowable error.

[0045] <<Actions and Effects>> As described above, the processing machine 1 according to the first embodiment processes the workpiece W in the following procedure. The control device 70 identifies a target shape based on the first depth image in which the sample S is captured. The control device 70 calculates the three-dimensional position of the surface of the workpiece W based on the second depth image in which the workpiece W is captured. The control device 70 calculates the three-dimensional position of the target shape when the workpiece W and the target shape are superimposed, based on the target shape and the three-dimensional position of the workpiece W. The control device 70 determines the cutting depth at each of multiple points on the surface of the workpiece W based on the difference in the line of sight between the three-dimensional position of the surface of the workpiece W when the workpiece W is viewed from a predetermined viewpoint (vertically upward) and the three-dimensional position of the surface of the target shape. The control device 70 moves the tool 60 based on the determined cutting depth. As a result, the processing machine 1 according to the first embodiment can process the workpiece in accordance with the target shape represented by the actual sample.

[0046] Furthermore, the control device 70 of the processing machine 1 according to the first embodiment calculates the depth of the surface of the target shape relative to the stereo camera 30 based on the three-dimensional position of the target shape, and determines the depth of cut based on the depth of the workpiece W in the second depth image and the calculated depth of the target shape. This eliminates the need for the processing machine 1 according to the first embodiment to convert the second depth image into three-dimensional data each time cutting is performed in order to determine the depth of cut. Specifically, once the control device 70 generates point cloud data in step S11 shown in FIG. 5 , it is not necessary to generate point cloud data in the loop from step S18 to step S31 or the loop from step S22 to step S31 for the selected stereo camera 30. The target shape of the workpiece W does not change from the start of processing to the end of processing. Therefore, if the control device 70 generates a target depth image and stores it in the memory unit 72 before machining in step S17, the control device 70 can use the target depth image stored in the memory unit 72 to calculate the depth difference in subsequent steps S21 and S27, thereby eliminating the need to convert point cloud data to a target depth image. Three-dimensional data is data representing the positions of multiple points in a three-axis Cartesian coordinate system. Point cloud data is an example of three-dimensional data. A depth image is data that associates depth with each point on a two-dimensional plane. Because depth represents the distance from the camera's viewpoint, it is not necessarily orthogonal to the two-dimensional plane. Because converting a depth image to three-dimensional data requires a large amount of calculation, the above procedure reduces the amount of three-dimensional data calculation, allowing the machining machine 1 to quickly machine the workpiece W. Furthermore, because three-dimensional data is large in volume, a large storage area must be reserved in the memory unit 72. However, using a depth image instead of three-dimensional data as in the first embodiment reduces the amount of data. In other embodiments, the depth difference may be calculated using three-dimensional data instead of the depth image, although this increases the amount of calculation.

[0047] Furthermore, the control device 70 of the processing machine 1 according to the first embodiment again determines the cutting depth for each pixel on the surface of the workpiece W based on the difference between the three-dimensional position of the surface of the workpiece W after cutting the workpiece W with the tool 60 and the three-dimensional position of the surface of the target shape. In this way, by recalculating the shape of the workpiece W each time cutting is performed, the control device 70 is able to constantly recognize the error between the actual shape and the target shape and cut the workpiece W while reducing the error.

[0048] Second Embodiment The processing machine 1 according to the first embodiment identifies a target shape based on a first depth image obtained by capturing an image of a sample S placed on a workpiece W. In contrast, the processing machine 1 according to the second embodiment captures only the sample S as the first depth image, and identifies the target shape based on this. The configuration of the processing machine 1 according to the second embodiment is the same as that of the first embodiment.

[0049] 7 is a flowchart showing a control method for the processing machine 1 by the control device 70 according to the second embodiment. The operator places a sample S on the table 10 without placing a workpiece W. When the operator inputs an instruction to specify the target shape, the control device 70 starts measuring the sample S.

[0050] First, the data acquisition unit 71 instructs the projector 31 to project a random dot pattern and acquires a depth image from the stereo camera 30 (step S101). This depth image is a first depth image that captures only the sample S. The target identification unit 73 generates point cloud data in the base coordinate system based on the first depth image and the position and orientation of the stereo camera 30 in the base coordinate system stored in the storage unit 72 (step S102). The target identification unit 73 merges the generated four point cloud data (step S103).

[0051] The target identification unit 73 calculates the height of the shape represented by the point cloud data obtained in step S103 (step S104). Next, the target identification unit 73 identifies the base shape by subtracting the height of the point cloud data obtained in step S103 from the shape of the workpiece W represented by the matching data stored in the memory unit 72 (step S105). In other words, the base shape represents the shape of the bottom of the workpiece W. Next, the target identification unit 73 rotates the point cloud data obtained in step S103 around an axis extending in the height direction and searches for a posture that fits inside the base shape when viewed from above (step S106). If there is no posture that fits inside the base shape (step S106: NO), the display control unit 76 displays an error message on the display indicating that the sample S protrudes from the workpiece W (step S107). The control device 70 returns to step S1 and waits again for an instruction to identify the target shape. The user considers changing the sample S.

[0052] On the other hand, if there is an orientation in which the target shape fits inside the base shape (step S106: YES), the target identification unit 73 generates target data by combining the base shape and the shape indicated by the point cloud data according to the orientation (step S108). The control device 70 waits for a processing start instruction (step S109). The subsequent processing is the same as that from step S10 onwards in the first embodiment.

[0053] In this way, the processing machine 1 according to the second embodiment can specify the target shape and process the workpiece W without placing the sample S on the workpiece W.

[0054] Other Embodiments Although one embodiment has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design modifications and the like are possible. That is, in other embodiments, the order of the above-described processes may be changed as appropriate. Furthermore, some processes may be executed in parallel. The control device 70 according to the above-described embodiment may be configured by a single computer, or the configuration of the control device 70 may be distributed across multiple computers, and the multiple computers may function as the control device 70 by cooperating with each other. In this case, the control device 70 may have some of the functions of the driver 58 of the robot arm 40, or the driver 58 may have some of the functions of the control device 70.

[0055] Although the processing machine 1 according to the above-described embodiment includes a stereo camera 30 as a depth camera, this is not limiting. For example, in other embodiments, a time-of-flight (TOF) camera, LiDAR, or 3D scanner may be used as the depth camera. Furthermore, in other embodiments, the processing machine 1 may include multiple single cameras instead of the stereo camera 30, and images captured by two adjacent cameras among the multiple single cameras may be subjected to stereo matching processing and used as a stereo camera. Note that, in other embodiments, the processing machine 1 may include only one depth camera. For example, in the processing machine 1 according to other embodiments, a depth camera may be provided directly above the workpiece W, and processing may be performed by pointing the tool 60 in the line of sight of the depth camera. Furthermore, in other embodiments, the depth camera may be a combination of a camera and a depth measurement device.

[0056] Furthermore, although the control device 70 according to the above-described embodiment converts the depth image into a base coordinate system to determine the cutting depth and create a path, this is not limited to this. For example, the control device 70 according to another embodiment may generate a tool path without converting the depth image by aligning the orientation of the tool 60 with the line of sight of the stereo camera 30. Furthermore, in another embodiment, although this increases the amount of calculation, three-dimensional data may be generated from the depth image to determine the cutting depth. When converting the depth image, the arm supporting the tool 60 does not need to have a high degree of freedom like the robot arm 40. For example, the arm may translate the tool 60 without changing the angle of the tool 60.

[0057] Furthermore, although the control device 70 according to the above-described embodiment identifies the three-dimensional position of the workpiece W based on matching data representing the shape of the workpiece W before machining, this is not limited thereto. For example, the control device 70 according to another embodiment may identify the three-dimensional position of the workpiece W based on the difference between the second depth image before the workpiece W is placed and the second depth image after the workpiece W is placed. Furthermore, in another embodiment, since the portion of the workpiece W held by the jig 20 is not machined, the three-dimensional position of the workpiece W may be identified and the target data may be aligned to the three-dimensional position by matching the shape (face, edge, etc.) of the held portion of the workpiece W shown in the second depth image with the shape of the target data. Furthermore, in another embodiment, the alignment may be performed manually by an operator.

[0058] Furthermore, the control device 70 according to the embodiment described above performs cutting for the second depth image captured by one stereo camera 30 until the depth difference becomes less than the allowable error, and then performs cutting based on the next stereo camera 30, but this is not limited to this. For example, in another embodiment, the control device 70 may switch the target stereo camera 30 each time cutting of one path is performed. That is, the control device 70 according to another embodiment may perform the processes from step S19 to step S27 for all stereo cameras 30, and then determine the end of processing in step S28.

[0059] <Computer Configuration> Fig. 8 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. The computer 900 includes a processor 901, a main memory 902, a storage 903, and an interface 904. The control device 70 described above is implemented in the computer 900. The operations of each of the processing units described above are stored in the storage 903 in the form of a program. The processor 901 reads the program from the storage 903, loads it into the main memory 902, and executes the above-described processing in accordance with the program. The processor 901 also allocates storage areas in the main memory 902 corresponding to each of the storage units described above in accordance with the program. Examples of the processor 901 include a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and a microprocessor.

[0060] The program may be for realizing some of the functions to be performed by the computer 900. For example, the program may be combined with other programs already stored in storage or implemented in other devices to perform the functions. In other embodiments, the computer 900 may include a custom large-scale integrated circuit (LSI) such as a programmable logic device (PLD) in addition to or instead of the above configuration. Examples of PLDs include programmable array logic (PAL), generic array logic (GAL), complex programmable logic device (CPLD), and field programmable gate array (FPGA). In this case, some or all of the functions realized by the processor 901 may be realized by the integrated circuit. Such an integrated circuit is also an example of a processor.

[0061] Examples of storage 903 include a magnetic disk, a magneto-optical disk, an optical disk, and a semiconductor memory. Storage 903 may be an internal medium directly connected to the bus of computer 900, or an external medium connected to computer 900 via interface 904 or a communication line. Furthermore, when this program is distributed to computer 900 via a communication line, computer 900 that receives the program may load the program into main memory 902 and execute the above-described processing. In at least one embodiment, storage 903 is a non-transitory tangible storage medium.

[0062] The program may also be a program for realizing part of the above-described functions. Furthermore, the program may be a so-called differential file (differential program) that realizes the above-described functions in combination with another program already stored in the storage 903.

[0063] According to the above aspect, it is possible to machine a workpiece based on an actual sample in accordance with a target shape represented by the sample.

[0064] DESCRIPTION OF SYMBOLS 1... Machining machine 10... Table 20... Jig 30... Stereo camera 31... Projector 40... Robot arm 60... Tool 70... Control device 71... Data acquisition unit 72... Memory unit 73... Target identification unit 74... Position measurement unit 75... Alignment unit 76... Display control unit 77... Difference calculation unit 78... Cutting amount determination unit 79... Path generation unit 80... Tool control unit 81... Correction unit S... Sample W... Work

Claims

1. A machine tool for machining a workpiece, a depth camera installed so that the workpiece enters an imaging range and imaging the depth of a subject, a target specifying unit for specifying the target shape based on a first depth image obtained by the depth camera imaging a sample representing the target shape of the workpiece, a position measuring unit for calculating the three-dimensional position of the surface of the workpiece based on a second depth image obtained by the depth camera imaging the workpiece, an alignment unit for calculating the three-dimensional position of the target shape when the workpiece and the target shape are superimposed based on the target shape and the three-dimensional position of the workpiece, a cutting amount determining unit for determining the cutting amount at each of a plurality of points on the surface of the workpiece based on the difference in the line-of-sight direction between the three-dimensional position of the surface of the workpiece and the three-dimensional position of the surface of the target shape when the workpiece is viewed from a predetermined viewpoint, and a tool control unit for moving the tool based on the determined cutting amount. A processing machine comprising the above components.

2. The processing machine according to claim 1, further comprising a projector that projects an image of an aperiodic pattern onto the imaging range of the depth camera, wherein the depth camera is a stereo camera and images the subject when the projector projects the image.

3. The first depth image is a depth image obtained by imaging the sample placed on the workpiece, and the target specifying unit specifies the target shape by removing the bottom of the three-dimensional shape in which the workpiece and the sample are integrated, which is specified from the first depth image, in the height direction. The processing machine according to claim 1.

4. The processing machine according to claim 3, wherein the target specifying unit specifies the target shape by removing the bottom in the height direction by the difference between the height of the target shape in the first depth image and the height of the workpiece.

5. The processing machine according to claim 1, wherein the target specifying unit specifies the target shape by adding the shape of the bottom of the workpiece under the three-dimensional shape of the sample specified from the first depth image.

6. A control device for a processing machine, comprising a tool for machining a workpiece, a depth camera installed so that the workpiece enters an imaging range and imaging the depth of a subject, and an actuator for driving the tool, the control device including: a target specifying unit that specifies the target shape based on a first depth image obtained by the depth camera imaging a sample representing the target shape of the workpiece; a position measuring unit that calculates the three-dimensional position of the surface of the workpiece based on a second depth image obtained by the depth camera imaging the workpiece; an alignment unit that calculates the three-dimensional position of the target shape when the workpiece and the target shape are superimposed based on the target shape and the three-dimensional position of the workpiece; a cutting amount determining unit that determines the cutting amount at each of a plurality of points on the surface of the workpiece based on the difference in the line-of-sight direction between the three-dimensional position of the surface of the workpiece and the three-dimensional position of the surface of the target shape when the workpiece is viewed from a predetermined viewpoint; and a tool control unit that controls the actuator based on the determined cutting amount.

7. A control method for a processing machine, comprising a tool for machining a workpiece, a depth camera installed so that the workpiece enters an imaging range and imaging the depth of a subject, and an actuator for driving the tool, the control method including: a step of specifying the target shape based on a first depth image obtained by the depth camera imaging a sample representing the target shape of the workpiece; a step of calculating the three-dimensional position of the surface of the workpiece based on a second depth image obtained by the depth camera imaging the workpiece; a step of calculating the three-dimensional position of the target shape when the workpiece and the target shape are superimposed based on the target shape and the three-dimensional position of the workpiece; a step of determining the cutting amount at each of a plurality of points on the surface of the workpiece based on the difference in the line-of-sight direction between the three-dimensional position of the surface of the workpiece and the three-dimensional position of the surface of the target shape when the workpiece is viewed from a predetermined viewpoint; and a step of controlling the actuator based on the determined cutting amount.

Images